Outpost Universe Forums

Projects & Development => Outpost 2 Programming & Development => Topic started by: BlackBox on February 13, 2011, 07:59:20 PM

Title: Git Server For Projects
Post by: BlackBox on February 13, 2011, 07:59:20 PM

Given the previous discussions that have been going on regarding splitting up SVN and hosting it on the main server, I decided to actually go a little further and setup a more modern version control system called git. You may or may not be familiar with it, and it is certainly different, but in my opinion it is a lot easier and better to work with than SVN.

First of all, I've set up a git browser/management system called gitorious and copied all of the existing SVN projects into it. You can check it out at http://git.outpost2.net/ (http://git.outpost2.net/). If you have ever used github, this is very similar although it's hosted on our own server. You can browse through projects here and see commit logs, get copies of the source code, etc.

As far as software goes, on windows, just how there is a graphical SVN program called TortoiseSVN, there is a similar one called TortoiseGit.

You need to install two packages, first Git itself: http://msysgit.googlecode.com/files/Git-1....iew20110204.exe (http://msysgit.googlecode.com/files/Git-1.7.4-preview20110204.exe)
Accept all the defaults in the installer, they should be sufficient for your needs.

You then can install the TortoiseGit shell extension: http://code.google.com/p/tortoisegit/downloads/list (http://code.google.com/p/tortoisegit/downloads/list) (pick 32 or 64 bit depending upon which type of OS you have).

If you're on another OS, or want more help (including pictures), go to http://help.github.com/git-installation-redirect (http://help.github.com/git-installation-redirect) and select your OS.

If you need help on actually using git (since its workflow is different from subversion) an excellent guide is here: https://git.wiki.kernel.org/index.php/GitSvnCrashCourse (https://git.wiki.kernel.org/index.php/GitSvnCrashCourse)

Also, if you want push access to any of the repositories here (pushing in git is the equivalent of committing in svn), please create an account on git.outpost2.net and then let me know; I'll add you to the appropriate group.

-- BlackBox

Title: Git Server For Projects
Post by: TH300 on February 14, 2011, 05:38:14 AM

Well, git is probably better. But unless you are familiar with it, it seems harder to use. when I do a "git clone git://git.outpost2.net/" I only get an error. So, I can currently not figure out the correct way to download the whole repository. Does that mean that I have to retrieve all projects separately? That would be a huge inconvenience.

The tarball generation takes forever.

As I am seeing, the svn version history was transferred and names of the commiters were preserved. So, we're obviously loosing nothing. So, we have at least one prerequisite for a complete move to git.

Altogether, I don't want to say that git is bad, but I'll have to learn a lot before I can use it correctly. Some guide specific to the opu git would be good. Maybe I'll write one myself, when I figured everything out.

Title: Git Server For Projects
Post by: Galactic on February 14, 2011, 10:53:31 AM

Thanks BB!

Title: Git Server For Projects
Post by: TH300 on February 14, 2011, 01:32:36 PM

Quote

Thanks BB!

Out of curiosity, why are you thanking BlackBox? As far as I know, you never commited anything to the svn repository and are not involved in any op2 related projects (unless you count the website itself as project).


Now, to clarify my concerns:

Many op2 coding projects use relative paths to include files from other projects (e.g. many missions include files from the sdk or the forced exports project). The structure of the svn repository is currently the correct structure that everything is "aligned to" and people don't have to worry about it since they get it automatically when doing a full checkout. Is it possible to work with the git repository in the same way?

Furthermore, you should decide on providing either git or svn, not both at the same time, since otherwise people will get confused about where to get the latest version of which project. (people could even start to make changes to a project in git and svn, what would mean a desync)

Edit:

You may want to change the ToS. They are currently specific to Gitorious.org

Title: Git Server For Projects
Post by: CK9 on February 14, 2011, 02:49:12 PM

Quote

Quote

Thanks BB!

Out of curiosity, why are you thanking BlackBox? As far as I know, you never commited anything to the svn repository and are not involved in any op2 related projects (unless you count the website itself as project).
 

Possibly because of potential issues that were being caused to the server(s) by the previous system?

I never commit anything to it, but I appreceiate the system as well, as I can actually read through it now in a way that doesn't make me think I'm missing something (and thereby causing me to give up on even trying to do anything with it).

Title: Git Server For Projects
Post by: alice on February 14, 2011, 03:44:55 PM

Quote

Thanks BB!

No, thanks for hosting it Galactic!

Title: Git Server For Projects
Post by: Arklon on February 14, 2011, 04:37:02 PM

Quote

Possibly because of potential issues that were being caused to the server(s) by the previous system?

The SVN was never on the OPU server, it was on Hooman's personal server (which is why you connected to it via an IP address rather than some outpost2.net address). This should also mean that uploading/downloading will be much faster than it was on the old SVN.

Quote

Furthermore, you should decide on providing either git or svn, not both at the same time, since otherwise people will get confused about where to get the latest version of which project. (people could even start to make changes to a project in git and svn, what would mean a desync)

As I said, Hooman personally runs the SVN, and he still needs to pull the plug on it. The Git server is intended to supercede it.

Title: Git Server For Projects
Post by: CK9 on February 14, 2011, 07:59:56 PM

Okay, then he could be thanking hacker for making it so he can help maintain it or something :P
 

Title: Git Server For Projects
Post by: Hooman on February 14, 2011, 10:01:48 PM

Yes, thank you BlackBox for taking the trouble of setting something up on a decent server. And also for the code BlackBox wrote but I committed (and hence is under my username). The "op2ext" project comes immediately to mind. I believe I've left a note about it somewhere. Check the project folders for a ReadMe.txt, (or perhaps the original commit log message). I think I left a ReadMe.txt with authorship information in a few cases where I committed code other people originally wrote.

I know there have been a few cases where code has been committed by someone other than the original author, particularly if a project is being forked or maintained by someone new. In that case, the original project is sometimes committed by the new maintainer as a base. Just keep in mind that it's generally polite to obtain permission from the original author first, and to leave a note about the original authorship, and that modifications may have been applied (and hence new bugs are not their fault).


I'll probably pull the plug on the SVN repository, or at least switch it to read-only or something for now. I guess I'd like to "git" used to git before I pull the plug completely.

And yeah, one reason why I hadn't switched to Git already, is it seems slightly more complicated than SVN, and I haven't been eager to force myself to learn something new. Hmm, perhaps I'm getting old. :(

Also, importing SVN stuff into Git isn't perfectly straight forward. There is usually some kind of glitch, at least if you're trying to preserve history. I think BlackBox said something about branches or tags not being marked? I haven't really checked out the new system yet.



In another slightly related note, the game server may move too at some point.


Edit: I too got an error when I tried to do a Git Clone.
fatal: http://git.outpost2.net/info/refs (http://git.outpost2.net/info/refs) not found: did you run git update-server-info on the server?

 

Title: Git Server For Projects
Post by: BlackBox on February 15, 2011, 06:10:10 PM

I suppose I'll post a quick tutorial on "how to use git" and how it differs from SVN (and some of the reasons I think it is better).

Git is considered a "distributed SCM," in that each user gets a complete copy of the repository (compare to SVN where the only thing the user has is the checked out working copy). This means you can make changes, commit them, etc. without affecting the remote repository at all (and means you can break the code in your repository as much as you'd like without anyone else having to see). It's a lot faster because of the lack of network access for most operations and this also allows it to be more intelligent when doing things like merges (you don't have to specify the revision numbers to merge like in svn, since git has the entire history and all branches available to it since you have your own copy of the repository).

The basic workflow is the following:

1. First you would obtain a copy of a project by cloning the repository. The usual git 'convention' for projects is to have a separate repository for each project or module (as you can see by looking around http://git.outpost2.net/) (http://git.outpost2.net/)). So for example, if I wanted to clone the SDK project I would do a git clone git://git.outpost2.net/outpost-2/dll-api.git (the exact URL to use is shown in the browser on the site). This is analogous to svn co <URL>.

2. Now you would have a complete copy of the repository in the dll-api directory. You can make changes to files in the repo like usual and then do git status to show the changed files; you add or mark files as changed using git add filename. (The nice thing here is, unlike SVN, you do not have to commit every file that changes, when you do git add they are added to a 'staging area' and you can continue to modify them and update the file in the staging area with successive git adds, or only add certain files you want added to the commit).

3. You would do a git commit to commit the files (you can also skip the above add step if you want to add all changed files to the commit with git commit -a). This is not exactly the same like SVN, as it does not go out the server at all; instead it records this revision into the local copy of the repository (so you can go back and look at the log of changes with git log and jump back to previous revisions and so forth). It would be comparable to having an SVN repository set up in a local directory and committing to that.

4. You would continue the above process of changing files, adding, and committing until you felt you had something you were ready to share with the rest of the world (git convention is to do commits for even very minor changes so you have more history in case you need to undo something, since no one else sees this yet).

5. Once you are ready to send your changes to the remote repository (i.e. the OPU git repository), you would do git push to transfer your changes to the server. Similarly, you can get changes made by other users by doing a git pull which is much like doing an svn up command.

Unlike SVN, git has support for real branches and tags; you can create a branch by doing git branch branch-name, switch between branches by doing git checkout branch-name, and merge changes from other branches into your current one by doing git merge source-branch. All of this is tracked and there are GUIs that can show the exact tree of branching and merging.

Additionally, the software we are using allows you to make 'private clones' of a repository that are stored on the server, if you want to make more extensive changes without affecting the mainline repository.

Now to answer some of your questions: there are separate repositories for each project, not one big huge one. As far as "sub projects" that referenced higher level directories in the SVN repository, you can get a similar behavior using the git submodule series of commands (basically this clones a separate repository into a subdirectory and treats that subdirectory like a symbolic ref to a specific version of the other project; for example this would make sense for libraries or for shared includes like the API). It would probably be a good idea to set up these projects in such a way (otherwise, there are some other tools that "wrap" around git as well that perform similar functions; one you may have heard of is called "repo" and downloads git projects into a certain directory structure as defined by a manifest; the Android open source project is an example of a use of this where around 160 different git repositories get combined into a monolithic directory structure and managed by repo).

In the end, I think git is a little different to work with (I wouldn't say hard per se since there are some pretty big advantages over SVN, IMO, it just takes a bit of a shift in thinking as opposed to the way SVN works).

Title: Git Server For Projects
Post by: TH300 on February 15, 2011, 06:43:15 PM

Thanks for the tutorial. Combined with the knowledge I gathered by hours of manpage reading I feel almost ready for using git. Well, I guess, I won't touch branches yet, but I will hopefully not need them.

Now, you told us a lot about basic git usage, but not so much about the Gitorious "interface". I was able to register as TH300 and hide my email address, hence no private information in the public so far. But when I do a local commit on my comp, my real name will be recorded (yes, I mind it more than Daniel and Garrett do). I found, that I can set the recorded name to TH300 by changing some configuration, but I am not yet sure, how I do this the best way.

Furthermore, I am wondering, how write access to the central repo is restricted. It seems to be done by requiring me to sign each commit with a private key (the public counterpart gets uploaded to Gitorious). Is that correct? Or is it enough if I just have a key configured when establishing the connection for a push (what would mean, the key is used like a password to gain access to the server)?

Title: Git Server For Projects
Post by: BlackBox on February 15, 2011, 08:02:12 PM

Oh, I suppose I should mention about committing/pushing to the server.

First of all, to answer your question about your real name going to the server, you can change the values it uses for this by doing the following:

Code: [Select]

git config --global user.name "YourNameHere"
git config --global user.email "your@email.here"

These can be set to arbitrary values and take effect for all usage of git on the system. You can test it out by trying a commit and you will be able to see from git log that it's recording these values (as an fyi, these settings get saved in ~/.gitconfig).

As far as pushing to gitorious goes, git doesn't have a special protocol, it just uses SSH. If you register in gitorious you need to generate an SSH keypair. You can use ssh-keygen -t rsa -C your@name.here (the email can be something fake, just something to identify you and hopefully matching the user.email you set with git config), this applies to people who are using Git on windows as well (do it from within the git shell). Don't set a passphrase (unless you want to type it everytime you do a push, since it's making an ssh connection). Then upload your ~/.ssh/id_key.pub to gitorious. But yeah, you're right, it uses SSH public key auth to authenticate you for pushing to the server.

Gitorious does some magic behind the scenes to check whether you actually have write access to a given repository and if not, it will send back an error when you try to push. In this regard the access control is probably a bit more granular than it was for SVN (I think you could only set permissions for the entire repository).

The other trick here, is that you have to push using the SSH url in gitorious (this won't show up unless you have write access to a repository, and the URL for pushing defaults to whatever you used to clone the repository). If you cloned using the git:// url instead, it's fairly straight forward to change that:

Code: [Select]

git remote rm origin
git remote add origin git@git.outpost2.net:project-name/repo-name.git

and then you can git push like usual.

Title: Git Server For Projects
Post by: Hooman on February 16, 2011, 12:58:30 AM

Quote

The nice thing here is, unlike SVN, you do not have to commit every file that changes ...

Perhaps I'm missing the real meaning here? I regularly do partial commits with SVN. If you're using Tortoise, just uncheck changed files that you don't want to include in the commit. If you're using the command line client, name the files you want to commit, or a folder to recursively commit all changed files. The default target is the current folder (which might not be the project root, and hence not contain all changed files).

Partial commits are nice when you catch bad code that is unrelated to the current change, and you want to update both at once, but them commit them separately. It works reasonably well if unrelated changes are in separate files. It's a little odd if you want to make two unrelated changes be two separate commits when they're both from the same file though.


One thing I didn't like too much about SVN, is you can't really work offline and build up a history of commits. Not without a lot of manual work anyway. Plus, I like how the full repository getting cloned is basically a convenient distributed backup system. Other than that, I haven't really noticed significant benefits of one over the other. At least not yet. Speed has never been an issue with SVN for me. Logs, status, and diffs appear to work without repository access. I haven't really done much branching or merging though, which I've heard may be better with Git.


Svnserve did have a way to set finer grained permission settings, but I never bothered with them. I've always found repository wide access controls to be sufficient and, naturally, easier to setup. No anonymous writes seemed good enough. Social conventions to dictate who should commit where seem to work, without stifling ability to contribute. If someone does misbave, they can't really delete things from a versioned file system, so the harm they can cause is somewhat limited, and at least undo-able.


So, is there an easy way to basically checkout/clone all the old projects in one go? I suppose this is why you mentioned repo? If so, is there going to be some kind of config file available for people that just want it all?
 

Title: Git Server For Projects
Post by: Zardox Xheonov on February 17, 2011, 08:35:46 AM

Thanks BB, I'll be sure to use git when i get home tonight. But for now I'm in study hall XD

Title: Git Server For Projects
Post by: onehand on March 28, 2011, 07:51:43 PM

This Git looks really interesting. Although, I wasn't able to register at  git.outpost2.net (http://git.outpost2.net/) because it never sent me a confirmation e-mail.

Anybody else have that problem? :/

Title: Git Server For Projects
Post by: Spikerocks101 on March 28, 2011, 08:26:15 PM

Works fine for me Onehand. Maybe you typed something wrong. I got my activation letter.

Title: Git Server For Projects
Post by: onehand on March 28, 2011, 08:50:09 PM

I had clicked back to double-check all the fields. I wonder if it's because of something weird like the hyphen in my domain name.

EDIT: Worked fine with a different e-mail address.

Title: Git Server For Projects
Post by: Hooman on April 23, 2011, 04:20:14 AM

Just in case anyone else has encountered this, I figured I should add a note here. I had cloned a few of the repositories, and quickly found out my local copies were showing modifications even before I'd touched the files. Turns out it was caused by line ending conversions. You can adjust the setting at:
TortoiseGit -> Settings -> Git -> Config -> AutoCrlf (uncheck)

You probably have to change the setting before doing the clone for it to do anything.

Also, I upgraded the version of TortoiseGit I had installed, and that seems to have cleaned up the local modifications part for the other repositories I had cloned.



I still haven't found a good way to clone a bunch of repositories all at once though. It's a very manual copy/paste link from website into Git clone command. It gets very tedious very fast.


Also, the level projects no longer link to the API projects very well. They are stored in separate repositories, and so there is no automatically implied grouping of projects so they can find each other easily. I assume there is some way of linking separate projects with Git, but I'm not familiar with it at the moment.