If you truly want to understand something try and change it.

I saw this quote on facebook last week and it really hit close to home.  I had just started working on a request at work to produce a flat file from a database I support that was to replace a similar file being produced by another team.  The other team was having to create a daily file, then at the end of the month concatenate those files into a single file only to have that file imported into a SQL table by a DTS job.  The DTS job had numerous SQL Tasks to cleanse, split, and sort the data for reporting for our business units.  It wasn’t an overly complicated process but there were about a dozen steps to get the end result and from what I gathered each month there were issues.  After spending several days reverse engineering the process I was able to eliminate the need for the flat file altogether.   I formatted the data in my query and was also able to eliminate the DTS as well.  The process now is a simple SQL Job with a step executing a stored procedure, with a second step to notify if it fails.  What was taking 30 minutes to import, cleanse, and split now only takes 4 minutes, most of that is network lag.  I now know more about this process than I ever cared to know only because I had to change it.  Moral of the story, if there is that process that you wish you understood better, just try to change it to improve performance, the end result will be a much better understanding of it.

Leave a Reply

Your email address will not be published. Required fields are marked *