Something was not right.
While doing bug fixes on the Online Inquiry App, queries were timing out. I upped the ADO.NET connection timeout to get around the problem, but that was a short term fix. Users can’t wait 30 to 60 seconds to get a page to load, at least not most users.
Have you ever been on the phone with a customer service rep that is waiting to pull up your record, and they say “Sorry, my system is a little slow today?” This is usually because performance is an afterthought for most developers. Especially those working at non-profit companies in the health-care sector.
I checked the queries. It was painful. The code was dynamic SQL, meaning that they were made up at run time, depending on what criteria the user selected. You did not know what was being submitted to the server unless you stepped through each call to the database. Sometimes the queries would be very fast, other times they would take almost a minute. And the pages would hang.
Oh, yeah, I’m sure there is some DBA pinhead out there reading this and saying “You could run a trace or use SQL Profiler to examine the queries! Gosh, you’re so stupid.” Well, fuck you, Mr. DBA. Most of the time you need special rights to run Profiler, and I don’t like to trace. So get back to your data-diddling, asshole.
Finally, I found an eight table join that was hitting a table with three million records. That is not a lot, but if you don’t leverage the indexes, you’ll be doing a table scan that brings the query to its knees. And some of these queries were hitting columns without indexes.
For the sake of keeping my development momentum, I thought about purging that table of all but 10,000 records. I fired up a delete query. As I was about to execute, I paused.
What if this was production data, I wondered?
After all, the database names weren’t prefixed by intuitive terms like “DEV” or “PROD”. I couldn’t put it past Mr. Whiteboard or Charlie to assume that these queries were read-only, and that it wouldn’t hurt to develop against production data. It’s not like some developer is going to go in there and blow out a few million records of live data, right?
Developers can, and do, purge tons of data in the course of real application development. That is why they need development environments. And QA environments. And Production environments with logins other than sa/password.
But for Cheap I.T. Bastards and Lazy Hacking Turds, having the discipline to not turn your production database into a sandbox is too much to ask. It costs money. It takes time. And it requires you to communicate – which is not a strong suite for someone trying to protect his turf or sabotage a new supervisor.
I was going to bring this up with Mr. Whiteboard. I closed the query window with the delete statement, updated the bug database and got ready to go home. It was Saturday, after all.