Split Svn Repository?

[Hooman]

Someone once told me (a long time back) a complete SVN checkout took a rather long time. I was somewhat surprised at the time since I'd forgotten about the game download folder and I didn't quite realize how large a complete checkout was. A complete export requires 127 MB of bandwidth, and 276 MB of disk space. An SVN working copy takes about double the disk space due to the cached original copies of the files in the hidden .svn folders. Of the 276 MB of exported data, 262 MB is accounted for by the game download folder. The size comes mainly from having multiple tagged versions of the download, where each version takes about 36 MB -38 MB.

I was thinking, would it be desirable to split the game download into it's own repository? That way you can easily do a checkout of the rest of the respository without worrying about or accidentally downloading multiple releases of the Outpost 2 game download? As it stands, you can do a partial checkout of just the parts you want, but this requires people to be a little more aware.

It would be fairly easy to split the repository in two and use addresses such as:
svn://68.146.111.58/opu  (all the normal projects)
svn://68.146.111.58/outpost2  (game download)
Or perhaps some other names that people find appropriate.


People that already have a checkout could probably split off the game download folder and run an SVN switch/relocate on each folder to point them at their respective new source repositories without needing to do another checkout/download.


Thoughts?
 

[TH300]

Splitting the repo would only make it more complicated for people who want to checkout everything. Yea, I'm one of those, because I like to have backup copies of op2 on my hdd.

In my eyes, people should checkout a subtree if thats what they want. The documentation seems to explain that (but I'm not sure, because I don't use Tortoise). We could possibly expand the documentation, so that it explains the directory structure of the repo. Also, a prominent note should be added to the documentation and warn the user about the game downloads.

And yea, the whole checkout takes long. The limiting factor seems to be your internet connection, since downloading 200 MB from other sites takes me less than a minute.

[Hooman]

I've also been thinking of moving the repository, but moving it elsewhere is not completely under my control. I guess the size would be less of an issue if the repository was hosted on a faster connection.


Hmm, any thoughts about using Git? I haven't really explored Git very much though, so I'm not sure what to think of it at the moment. It sounds like it might have a few advantages over SVN. Honestly though, I've been pretty happy with SVN, so I'm not sure if it'd be worth switching over.
 

[Sirbomber]

Why is the game even on the SVN?  The only people who actually use it won't need a game download up there.

[Hooman]

Patches. You'll notice it starts with a CD install, and proceeds with the 3 official updates, and then the custom OPU patches. It's a project being tracked in version control like any other. It's just much larger than the rest and consists of mostly binary files.
 

[alice]

If you tell it to download the entire repository, that's what it will do. Maybe an organizational change would resolve the issue, if it's not clear what you are downloading? Or, if someone downloading the entire repository by accident was the result of the SVN tutorial that you provided, maybe it needs updated and clarified..

But, having each version of the client patches within the same repository might be a little silly.. I think that's a justifiable case for splitting it out, "maybe". Having them combined allows for moving them within the same repository, should we ever choose to combine them again (or if the code-based patches start to become their own "project" and they get moved out to their own project.), they can be moved within the same repository without losing historical version information. (although, seeing that the distributions are mostly binary, you're effectively keeping multiple copies of each file in the repository.. perhaps having the versions expanded and then soft/lazy copied would be preferred (unless they're already set up like this)

I'd lean toward keeping them in one repository; If it's organized correctly and the people that are new to SCM are informed about what the commands they issue might do, then there probably shouldn't be an issue.. Although I can't say I've used the repository much, so I'm not sure if this is exceedingly relevant to the current repository.

Regarding the hosting, it's likely that we can run SVN off of the outpost2.net server; We don't have Apache mod_svn/DAV enabled AFAIK, but a straight svnserv daemon running would likely suffice, and it would be in a location that is "always up" and has a fast connection.

[Hooman]

I'm not too sure what you mean about each version of the client patches. Essentially the game download is a project, and the tags are the different released versions. That is fairly standard versioning practice. The question is more about whether to include the game download project with the rest. It's much larger than the rest, and usually requires modifying binary files and disassemblers to update it, so it's more involved to work on that project. It's also perhaps more closely scutinized when a revision is committed to that project, since that stuff is occasionally packaged for download on the main site, and we don't want quality or security problems with it. They're all Outpost related projects, just that one is significantly different from the rest in terms of size, required skills, quality, and security concerns.

Also, SVN uses a binary diff algorithm, so it doesn't matter if the files are binary or text based. Storage requirements on the server for the different versions is proportional to the size of the patch, not the size of the files being patched. If you check out a working copy of all versions, then of course that will require size proportional to the complete file size. That's more where the issues lies. I'm uncertain if the bandwidth requirements to checkout two nearly identical versions is the size of both versions, or the size of the earlier version plus the size of the diff to the newer version. When I did an export, the total bandwidth usage seemed lower than I expected, so it may use the diffing approach, but it also wasn't quite as small as I'd hoped.


I'm leaning towards having appropriate usage documentation and hosting on a faster connection as the best solution. The idea of running the SVN repository on the web server machine has come up, but I'd want to get an OK from BlackBox before going ahead with a plan like that. I was thinking svnserve rather than using an Apache module.
 

[Zardox Xheonov]

as long as it makes it easier in terms of using the svn, then im all for it.

besides that this sounds like it would make organising easier or at least waste less mb space on hardrive/disk so overall im for it.

[TH300]

as long as it makes it easier in terms of using the svn, then im all for it.

besides that this sounds like it would make organising easier or at least waste less mb space on hardrive/disk so overall im for it.

A switch as Hooman suggested it would really only change the interface. Right now you have the option to download only the parts that you want (and save disk space), hence no change in functionality. And the interface wouldn't even be simplified, it would be complicated for some users: If I want everything I'd have to checkout from two or more repositories.

[Hooman]

I think the best solution is probably to just host the repository on a faster connection. The organization can stay as it is, and a note can be added about partial checkouts.

I'll probably have to organize something with BlackBox about it.