Kung Fu for SysAdmins: Using Version Control Systems for scripts

Like nearly every administrator I know, I have a few scripts that I use to help me do my job easier and faster. Well ok, more than a few. Alright, alright, I have more scripts than most people have hair.

I have scripts that were written for an NT4 Alpha Cluster. I never get rid of them. I’ve lost more scripts than most people will ever have. I’ve forgotten more about most scripting languages than most people will ever know.

I find that the challenge is keeping track of them. If I could turn my computers, network storage, USB drives, and email upside down and shake them, enough scripts would fall out to fill the library of congress. Damned if I know where they’re all stored, but I know I have them.

Unfortunately, most of them are variants on the same script, or actually are the same script. Many times if I can’t find the exact script I am looking for, I’ll slap one together, do some quick testing, give it a unique name (you know something that makes it easy to know what it does, like “test74.vbs”), use it once or twice, and then forget what it was for.

Yes I’ll admit it, I have a problem. If there were an AA for scripters, I’d probably be the president of the local chapter.

Or at least I would have been before I started using Version Control software. About a month ago I was editing a script that I was having troubles with, accidentally overwrote something that broke it, and closed the editor. Not a big deal on small scripts, but this one was at about 1500 lines. It took me HOURS to figure out exactly where the code was that I’d overwritten, but I never did figure out exactly what I did that broke it (though I did get it working well enough to do what I needed it to do).

I’ve known about Version Control software for years (I worked at a software company once upon a time), I’ve even used Subversion before. I’d been meaning to set it up for myself for years, and this incident was the catalyst I needed to actually do it.

Keeping the LAOAE principle in mind, I wanted my repository to be available to me in all the places that I’d be likely to be working on a script, mostly at work and at home.

But I already have enough servers to admin, and I don’t really want to have to care for and update yet another thing on my work network, so I started looking for hosted subversion offerings. I’m not a software company, so I had to weed out the ones that wanted absurd monthly fees (and offered absurd feature sets). I just wanted hosted Subversion, I didn’t need team collaboration, or project management features. I thought about going Open Source with it (like github, or Google code) but decided that since I would be using it to also host code that belongs to my employer, it would probably be better to go with a commercial solution.

Did I mention that I didn’t really want to spend any money on this? Yeah, free is king in the land of the Sysadmin. There were several services that fit the bill, in the end I decided to use ProjectLocker. Their free offering gives you three users and 300MB of storage (even for my bloated script collection this is plenty) three Repositories, and three Projects (each project can have an unlimited number of files and folders), though for only $19/month you can move up to 15 users and 10GB of storage (full details of their offerings can be found here).

Oh and all of their plans offer both Git and Subversion, so whichever you are more comfortable with is available. I have used subversion in the past, so that’s what I chose to go with.

Now before I get any further in this post, I am going to write this with the assumption that you have at least a conceptual knowledge of how Version Control Systems (VCS) work. If you don’t and would like to, this free ebook is a great place to start (and really relevant, as it’s also the official documentation for Subversion).

Setting up a VCS for use with scripts

ProjectLocker (Subversion)

Initial setup of a ProjectLocker account is quick and easy:

  1. Select your Service Level (Free is fine for me).
  2. Select you term (Free is Always Free).
  3. Enter a promotional code (optional).
  4. Enter a Referral Source email address (if you have one, the referrer gets free storage space in small increments).
  5. Click “Next Step”.
  6. Agree to the Terms of Service (if you do), and select the “I am ready to setup my account” button.
  7. The information on this page is pretty self explanatory, click the “Place My Order” button at the bottom when done

Save the Login URL on the resulting page! This is the URL you will use to access your repository (https://portal.projectlocker.com).

Log in to your repository and fill out the requested information (it helps keep the free offering available).

That’s pretty much it on the Subversion side, though you can set up users and additional projects if you like. Since I am using this to host my personal code, as well as code that belongs to my employer, I set up two projects: Personal_code and Work_code (I know, original right?).

Once you have a repository running, ProjectLocker will give you a URL to the repository, it’ll look something like this: https://pl3.projectlocker.com/TestCompany77/Personal_code/svn

This is what you’ll need to access the repository from a client, so write this down (bookmark it, whatever you have to do, you will need this)! (obviously use the one in your account, the one listed above won’t work for you)

I also set up a user account for the main IT email account at the office so that it has access to the Work project only. This way when or if I leave my current job, I can just hand off the login to the next person (or leave it with someone here) and they will be able to log in and access all of the scripts that are owned by the company, including being able to see all the changes I’ve made, and any comments I’ve made during the commit process.

TortiseSVN (subversion client)

So now that we have a repository, we need a subversion client. On Windows that leads us to TortiseSVN. There are others, and you are certainly welcome to use a different subversion client,  but TortiseSVN is the hands down leader of the pack for features and maturity. It’s also really easy to install and configure.

TortiseSVN is a command line interface to subversion, but it integrates with the Windows Shell. This means that there is no “program” window to access for settings and such, you access everything via context menus (right click).

Once you have it installed on a Windows computer, you’ll need to link it to your Repository. The easiest way I’ve found to do this is to use the TortiseSVN Repo-browser (you’ll want to get familiar with this tool anyway, as it is installed with TortiseSVN by default and you can use it to… well browse your repository).

Just right click on any file or folder, highlight the TortiseSVN entry, and select Repo-browser from the resulting menu:

TortiseSVN context menu

In the Repo-browser window enter the URL for your subversion repository and click the OK button:

When the Repo-browser attempts to connect to your repository, it will ask you for a username and password, and if you want to store that credential in a file on your computer (that’s up to you):

Once you have entered the credentials, it will show you the contents of your repository:

Now TortiseSVN is ready to use! The Repo-browser interface is fully drag and drop, so you can just drop your script files and folders on the right pane, and it will upload them to subversion. You’ll need to enter a commit message, and once the upload completes, you have your first version of your files in subversion.

NotePad++ (text editor)

Now all we need is an editor that can make use of this, and we’re in luck. Perennial favorite NotePad++ has an extension for TortiseSVN. First, you’ll need to install NotePad++ (or open up the portable version on your USB drive). Once you have NotePad++ open, Select the Plugin Manager from the Plugins menu as shown here:

One of the reasons that I’ve been such a big fan of NotePad++ is it’s extensive selection of available plugins. In the Plugin Manager, just select the Tortise SVN plugin and click Install as shown in the image below.

This plugin requires two supporting plugins, which will automatically be selected for install, as shown here:

Once the installation is complete, NotePad++ will need to be restarted (it will tell you this and prompt you for the restart of the program). After NotePad++ is restarted, we are ready to start actually using our version control system!

If you’ve never used a VCS before, you’ll need to understand the concept of Working Sets and the Checkin/Checkout relationship. These topics are far outside the scope of this post, but if you are new to this, I’d really suggest reading the official Subversion documentation (or at least skimming it). You can find the official Subversion book here.

In order to work on any files stored in the repository, we first need to create a local copy of the repository, or a “Working Set”:

  1. Create a folder where you want to store your Working Set. You can name it anything but “svn”, as this name is reserved for use by TortiseSVN. I usually choose something like C:\SOURCE, so it’s easy to remember.
  2. Right click on the folder you just created and choose SVN Checkout from the context menu.

At this point, TortiseSVN is going to prompt you for some information about how you want the Working Set created, in the form of this window:

The only thing you really need to be concerned with here is the Revision section. You only want to change this if you don’t want the latest revision of the files in your repository (which is what HEAD revision means). If you need an earlier revision, select the Revision radio button, and then use the Show Log button at the right to search for the desired revision. If this is the first time you’ve set this up, or you want the latest revision, just click the OK button. After all the files from your repository are copied, you’re ready to start editing!

Up to this point, everything we’ve discussed has been pure configuration, something that you’ll do one time per computer, and that’s it.

Using the Version Control System

Here is an example of the typical workflow of editing a script without using a VCS:

  1. Find the script (this is usually the hardest part).
  2. Edit the script.
  3. Save the file.
  4. Pull your hair out because the 2500+ line script you just changed isn’t working right (hopefully you’ve never experienced this step).

Now here is an example of the typical workflow of editing a script using the VCS we just set up:

  1. Open the file you want to edit from the Working Set in NotePad++. For this example I’ll use the file C:\Admin\Scripts\SOURCE\Shell\reboot.cmd from my local Working Set.
  2. Make your changes in NotePad++, and save the file. If you do not save the file, there is nothing to commit to the repository.
  3. From the NotePad++ Plugins menu, select Tortise SVN > TSVN – File commit, as shown in this image:

This tells TortiseSVN that you want to write the changes you’ve made to your repository, and it will prompt you to do so using the Commit window:

As you can see there are several options before you actually commit the file. In practice you’ll usually only need the Message area and the OK button. The Message area is basically a comment area for what these changes represent. The more verbose you are here, the easier it will be to understand the changes that were made in the revision, and it will also make it much easier to find a revision where a particular change was made. Once you click the OK button, TortiseSVN will commit your changes to the repository, as shown in this window:

Clearly you can see this is a slightly more complicated process, but the pay off totally worth it if something goes wrong.

How to figure out where you broke something after you’ve been using this for a while

The exact workflow that any given person uses will probably be slightly different than that used by any other person, so this is going to be a description of my particular workflow, and how I track things.

I’m writing a PowerShell Module (sshhh), and I have a function in it called Get-Sysinternals. When I first wrote this function, it would default to downloading all of the Sysinternals utilities. I’ve edited it and altered it several times since then, and I wanted the default behavior to be that it would only download updates to the tools if they were already installed on the local computer in a specific folder. I then later added some switches to change the default behavior, and somewhere along the line, I broke the default behavior.

To figure out what changed, I needed to see what the code looked like when the default behavior was changed. To do this I opened the file (AdminsArsenal.psm1) in NotePad++, and from the NotePad++ Plugins menu, I selected Tortise SVN > TSVN – File log.

This brings up a viewer for all of the Commit activity for the file, which looks something like this:

Scrolling through the commit messages, I find that at revision 76 I changed the default behavior.

What you’ll do at this point depends on how you want to handle this. You could right click the file in explorer, and select TortiseSVN > Update to revision…, if you just wanted to revert the file to a point where you know it worked. Personally I just wanted to see the code in revision 76, so I right clicked the revision I had highlighted, and selected Compare with working copy from the resulting context menu:

Now this particular example was not a great one to use, as I completely rearranged the functions in that module in a later revision so the compare is pretty sloppy, but you get the point.

You’ll also notice that in the Log Messages window, it shows the Author which makes it really easy to determine who made what changes to a given file if you have more than one person working on your scripts.

%d bloggers like this: