On using Git submodules

We at Feinheit had been using Git submodules to include 3rd party apps into our Django projects and have now decided to switch over to using Virtualenv and pip instead. The reason for this? Because 'everyone was doing it'.

Using Git submodules to keep track of - well - submodules

While pip is really easy to use, there is one downside: You don't have access to the source of the app. Which is fine for apps that just work. But we have some apps that we are using in multiple projects which are still under heavy development and changes and improvements are made all the time. For those app there is another solution: Git submodules.

The idea behind this is that Git keeps a list of repositories that will be included as children of the main repo. For each app Git saves the source path, destination path and commit ID. This way you can change the included app without braking the main app because it will stay at the specified commit.

Of course you need to be using Git as your version control system to be able to use submodules.

Submodules are easily included. It is a bit trickier to remove them though. So be sure to include them correctly.

To use submodules, it is best to have a lib folder to store all the modules and their docs.

Usage example

In our example I have a project and would like to include the django-scaffolding app. But because I know I want to make changes to it, I create my own branch of it on github.

In my own project directory (git root) I can add the submodule with the following command:

git submodule add git@github.com:sbaechler/django-scaffolding.git lib/django-scaffolding

The first argument is the url to the repo on github. The second argument is the target directory. I'll place it in my 'libs' directory. Now I just create a symlink, so Django can find the package:

ln -s lib/django-scaffolding/scaffolding

I now have to add that folder and the new submodule to the main repo:

git add scaffolding
git add lib/django-scaffolding
git commit -m 'added submodule django-scaffolding'

That's it. I can now add 'scaffolding' to settings.INSTALLED_APPS and can use it. And it's right there in my project dir.

When I push this repo to the live server, I can install and update all submodules with this command:

git submodule update --init

Make sure the live server has access to github.

Making changes

After I have made some changes I would like to commit them back to github and also to the version that is on the server.

First you have to update the scaffolding repo:

cd /lib/django-scaffolding
git add -u
git commit -m 'Your message here.'
git push

Then you have to update the commit id in the main repo:

cd ../..
git add lib/django-scaffolding
git commit -m 'update in django-scaffolding'
git push

Now you can use git submodule update on the live server to pull in the changes.

Note that when you use 'git submodule update', git will point to a commit, not to a branch. Before you make changes make sure you checkout a branch first.

Next time I will show you how to remove a submodule from a project.