Symphony.

I've read (bunch of times :-) ) the Getting Git for Symphony and I'm doing the "Welterweight: using a complete site repository" with a Private repository that I've setup. Everything seems to be working ok. I managed create the bare repository, setup my local development directory, push that to the Private repository, then clone and pull to a Staging domain. I've also read the using git for private development and symphony, which really helped too. At this point I'm just manually updating the database after the install.

Where I'm running into trouble is how to handle the Extensions. I noticed that I need to init and update the extensions on my Staging domain, they don't come over when I clone the Private repository. The extension directory and sub-directories are on Staging, but they're empty. When I browse to Extensions in the Symphony admin I get an error: "Could not find extension at location ... " Once I do the init and update they'll work. I have to do the 2-liner with my host.

git submodule init
git submodule update

I feel like I'm doing something wrong. I was thinking it was a complete site repository outside of the files in the .gitignore, that would include the extensions. I've removed the ignore on the workspace directory because I want those files to move over. After trying a few times I see that now when I do the clone on the Staging I'll get the additional Extensions that I added locally too. So that seems to work as long as I do the init and update. Maybe I'm not that far off from where I'm going.

I need to read up more on submodules I'm sure... (I'm pretty new to GIT so it's been a bit of struggle, I'm sure I was thinking about it in my sleep). In the end I have a feeling I might use SVN and Cornerstone to version control my workspace, but I wanted to get some learning down on the git approach. I've managed to put together a little technical write-up for myself so when I come back someday it'll make sense. I'm just getting a little hung up on the Extensions at this point.

I'm loving Symphony! (and this is an awesome community that I hope to contribute to eventually!)

What you're experiencing is the main point of submodules: Minimizing repetition and keeping the histories of the repositories independent.

You're not downloading and committing the actual files of a submodule into your Symphony-repository but only what's necessary to know where to pull those additional files from (the submodule-URL) and what to pull (the commit ID your submodule is set to be).

That way you don't have to merge all changes to the submodules into each and every project that's reusing them. Instead you'll maintain and update one submodule-repository (a particular extension) and all repositories using it (your Symphony installations) can update to the latest version easily and with one single commit (in most cases annotated with a commit message like "Tracking latest version of JIT image manipulation").

In short: You always have to run git submodule update --init after cloning.

Like all of my professional projects, I copy the specific releases of extensions into my site's repository. I find this guarantees that I'll always have a consistent, always accessible version of my Symphony install. I got bit too many times by people moving or renaming their repositories, and I've found got submodules to be fickle at best, and downright annoying at worst.

I do the same for my mac and iOS Xcode projects. Having a stable, unchanging and properly versioned copy of my project is more important than saving me copying a few files once in a while.

I copy the specific releases of extensions into my site's repository

I do the same. I don't want a project to have a reliance on a third party submodule. If the developer releases a new version that is buggy or incompatible with my Symphony version, the submodule will update and break my site. So I don't use submodules, but instead clone an extension into my site then kill the .git directory before adding the files to my main repo.

I think Nils has a more complex way that involves creating a separate repo for the /extensions directory, and forking all extension repos, so that they remain decoupled from the third party. But I may have dreamed that.

I have used submodules for coupling internal projects (where one project comprises multiple private repositories), but never use them for third party public extensions.

Has anyone else been burned by this, and have a workaround?

But I may have dreamed that.

No, that's exactly our setup.

By the way, I don't think submodules update automatically - they only do so on request as far as I know.

Can you therefore update a single submodule using git if you knew the extension was tested compatible? tried a submodule update and got burned with one or two submodules breaking like Nick mentions.. it was a dev project so no biggie!

I understand in theory why you would not use submodules for extensions but I have never experienced issues in practice (apart from issues with deployment, etc).

Git submodules do not update automagically, you'd have to manually run a git pull right?

I don't see why this could cause issues, as long as one is careful and not 'just' pull all submodules. The good thing about using Git (submodules) for extensions is the ease-of-updating (if you want) and the ease of a rollback (i.e. most benefits from Git in general).

I don't see why this could cause issues

My example was developing a project locally for about four months, then deploying to the live server directly from git (and pulling the submodules), but the submodule repo had since been updated by the third party, so the extension was different locally (which I hadn't updated since adding it as a submodule) than on live.

Aha! Good point. Since I deploy using deployHQ I a have not yet run into this because they deploy using FTP.

You are using Git on the server and, therefore, with the first deployment will 'fetch' the submodules (an potentially break stuff when submodules are newer/incompatible)

This will only occur the first time you deploy using Git though, right?

One way around this would be to make sure you attempt to update all submodules locally (and fix issues) right before deployment. This is not a strange scenario, imho, because you'd run into issues updating the submodules locally later anyway. - But again: good point (that I will keep in mind when I try to look into Git-deployment :) )

I am solely relying on submodules for my extensions and never had any problems. The only thing that might cause trouble is when people delete/rename their repositories. But that's no biggie to fix either.

What git submodule update does

First misconception you seem to be having is that submodules update at random or you could never be sure what version you get. The contrary is true: submodules always save what commit your submodule is set to and they only attempt to checkout that specific commit. For example clone the Symphony main repo and execute

> git submodule

You will see something like

 07a354c5761f2405479eff5c9ac099a3b3fa2d2b extensions/debugdevkit (1.2-1-g07a354c)
 db1fdd53f3c1551bf5b28c777f35534e2fe6441a extensions/export_ensemble (1.10-36-gdb1fdd5)
 1862e9ab66e8fb3cc8ef5eceacceaf40527da30b extensions/jit_image_manipulation (1.09-63-g1862e9a)
 8b5ab1391394242cc38a095384486cdee3ef2e4f extensions/maintenance_mode (1.2-16-g8b5ab13)
 ...

Each of those lines is a submodule; the first part is the commit that will be pulled, the second part is the folder it will be pulled into.

So now every time you do git submodule update, all the submodules mentioned in that list, will be updated to those exact commits. That's also the case when you did a clean clone and are doing the git submodule update --init afterwards.

If you know there was an update in one of your submodules doing a git submodule update will not update anything. It will only revert to what your superproject is set to be tracking. Instead you have to go into the submodules folder and do a git pull origin master there.

So when used correctly, two clones of a project never ever end up having different submodule versions.

Submodule-URL changed

Like I said the only issue I've experienced was when public extensions disappear or their URL changes. Again, no biggie. If the repo has been renamed, go into .gitmodules and change the URL there too and afterwards do a

> git submodule sync
> git submodule update

Now all the remotes in your submodules are set to the respective value in .gitmodules.

Submodule-remote disappeared

It's a bit trickier when the owner of the submodule has disappeared and the whole repository is gone altogether.

In this case you have to simply re-create that public repository i.e. using your GitHub account:

Since you already have a perfect clone of that repo go into the submodule, add your new GitHub repo as a remote there and do a push. Done, you now have a as good as new clone of the submodule you were using. Continue with the steps mentioned earlier.

Thanks phoque, that explains a hell of a lot for me!

Yeah, nice post. I only just discovered git submodule sync a few weeks ago. I wish I had known about it earlier when I was manually editing the gitmodules file...

Oh, and if you want to stay on the bleeding edge: git submodule foreach git pull will update ALL of your submodules to the latest version. Don't do this on a production site of course :)

git submodule foreach git pull will update ALL of your submodules to the latest version

Very cool.

@Henry, thanks for those tips. I did not know you could do that. Very cool indeed.

The above from Henry also works if you want to checkout the master branch of each submodule in one go!

git submodule foreach git checkout master

Encountered a situation when cloning a repo with submodules where the gitmodules file was ignored and when I added it back in the above from henry didn't work cos the submodules werent tracking an actual branch for some reason! git is cool :)

All very cool, yes! Thanks this really helps!

If you know there was an update in one of your submodules doing a git submodule update will not update anything. It will only revert to what your superproject is set to be tracking. Instead you have to go into the submodules folder and do a git pull origin master there.

So when used correctly, two clones of a project never ever end up having different submodule versions.

That's good to know along with :)

In short: You always have to run git submodule update --init after cloning.different submodule versions.**

Not sure if people knew this one (probably most do):

git clone git@github.com:whatever/repo.git .

Using this when you're inside the folder you wish to clone a repo into, will pull the contents of the repo and not the folder as well. the . specifies current folder location as the checkout folder..

Note: the folder needs to be empty to do this... so if you have a .git folder in there it needs to be removed first.

Whoa. This thread is gold.