[Git] [Step 1] Update autobuild utility for git workflow #1819

New Issue

Open

opened 2013-01-12 21:23:33 +00:00 by historic_bruno · 11 comments

historic_bruno commented

2013-01-12 21:23:33 +00:00

Currently the Windows autobuilder works with our SVN repository (see BuildServerSetup). As we're transitioning to git, we need it to work with a git repo instead.

support multiple git branches, both the "master" and various feature branches, so e.g. artists can easily test new features
commit a file for the relevant branch, that can be tested by an updater utility, and contains paths and checksums of each file that was built
the built files will be uploaded to a web server

See also #1814 and #1816.

Currently the Windows autobuilder works with our SVN repository (see [BuildServerSetup](BuildServerSetup)). As we're transitioning to git, we need it to work with a git repo instead. * support multiple git branches, both the "master" and various feature branches, so e.g. artists can easily test new features * commit a file for the relevant branch, that can be tested by an updater utility, and contains paths and checksums of each file that was built * the built files will be uploaded to a web server See also #1814 and #1816.

historic_bruno added the

Build & Packages

1: Release Blocker

labels 2013-01-12 21:23:33 +00:00

historic_bruno added this to the Alpha 13 milestone 2013-01-12 21:23:33 +00:00

Ykkrosh was assigned by historic_bruno

2013-01-12 21:23:33 +00:00

historic_bruno commented

2013-01-13 20:21:48 +00:00

Author

How will we clean up old autobuild files, e.g. feature branches that have been merged into the development branch and removed?

If I'm correctly understanding the proposed design, it will be like this:
<http://foo.wildfiregames.com/autobuild/master/>

Then someone creates a def-lighting feature branch and autobuilds it:
<http://foo.wildfiregames.com/autobuild/def-lighting/>

We approve it and it gets merged into master, what happens to the def-lighting directory? Should the autobuilder itself be responsible for checking the branches and pruning old ones?

How will we clean up old autobuild files, e.g. feature branches that have been merged into the development branch and removed? If I'm correctly understanding the proposed design, it will be like this: `<http://foo.wildfiregames.com/autobuild/master/>` Then someone creates a def-lighting feature branch and autobuilds it: `<http://foo.wildfiregames.com/autobuild/def-lighting/>` We approve it and it gets merged into master, what happens to the def-lighting directory? Should the autobuilder itself be responsible for checking the branches and pruning old ones?

k776 commented

2013-01-13 20:43:24 +00:00

This is what I had in my head: The auto builder needs to know which branches it can build, so it needs to either maintain a git repo and get the branch list from that OR use the github API to get the branch list. Either way, for it to see that a branch is gone, we'd trigger an update of the autobuilder branch list. When it sees a branch is gone, it disappears from the list and removed the folder for that branch.

This is what I had in my head: The auto builder needs to know which branches it can build, so it needs to either maintain a git repo and get the branch list from that OR use the github API to get the branch list. Either way, for it to see that a branch is gone, we'd trigger an update of the autobuilder branch list. When it sees a branch is gone, it disappears from the list and removed the folder for that branch.

k776 changed title from ~~Update autobuild utility for git workflow~~ to [Git] [Step 1] Update autobuild utility for git workflow

2013-01-22 22:24:52 +00:00

historic_bruno commented

2013-02-09 23:01:59 +00:00

Author

See also BuildAndDeploymentEnvironment#Futuredesign, I updated it to our current plans.

See also [BuildAndDeploymentEnvironment#Futuredesign,](BuildAndDeploymentEnvironment#Futuredesign,) I updated it to our current plans.

fabio commented

2013-02-10 10:34:37 +00:00

Why under master branch the "trunk" directory and not directly its subdirectories?

Why under master branch the "trunk" directory and not directly its subdirectories?

historic_bruno commented

2013-02-10 20:46:50 +00:00

Author

Replying to fabio:

Why under master branch the "trunk" directory and not directly its subdirectories?

Good question :) I corrected that and improved the formatting a bit.

Replying to [fabio](https://gitea.itms.ovh/wfg/0ad/issues/1819#issuecomment-225024): > Why under master branch the "trunk" directory and not directly its subdirectories? Good question :) I corrected that and improved the formatting a bit.

fabio commented

2013-02-11 11:23:41 +00:00

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: (http://trac.wildfiregames.com/ticket/304#https://gitea.itms.ovh/wfg/0ad/issues/1819#issuecomment-225023)

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad. I already suggested this here: (http://trac.wildfiregames.com/ticket/304#<https://gitea.itms.ovh/wfg/0ad/issues/1819#issuecomment-225023>)

quantumstate commented

2013-02-23 15:21:35 +00:00

Replying to fabio:

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: (http://trac.wildfiregames.com/ticket/304#https://gitea.itms.ovh/wfg/0ad/issues/1819#issuecomment-225023)

We want to try and keep the process as simple as possible for the main development team. Having to update two repos goes counter to this and will almost inevitably lead to someone wasting time because they forgot to update one and now hit an issue. A small amount of extra downloading for people wishing to use the engine isn't so bad. Also many mods will want to use our javascript which is part of the public mod anyway.

Replying to [fabio](https://gitea.itms.ovh/wfg/0ad/issues/1819#issuecomment-225026): > Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad. > > I already suggested this here: (http://trac.wildfiregames.com/ticket/304#<https://gitea.itms.ovh/wfg/0ad/issues/1819#issuecomment-225023>) We want to try and keep the process as simple as possible for the main development team. Having to update two repos goes counter to this and will almost inevitably lead to someone wasting time because they forgot to update one and now hit an issue. A small amount of extra downloading for people wishing to use the engine isn't so bad. Also many mods will want to use our javascript which is part of the public mod anyway.

historic_bruno commented

2013-02-23 21:02:33 +00:00

Author

And anyway there's not a clear division between engine/mod/data currently, so until (if) we make that clearer, it's not worth worrying about separating them in different repos. As-is, source/ wouldn't be usable without binaries/data/.

And anyway there's not a clear division between engine/mod/data currently, so until (if) we make that clearer, it's not worth worrying about separating them in different repos. As-is, `source/` wouldn't be usable without `binaries/data/`.

k776 modified the milestone from Alpha 13 to Alpha 14

2013-03-15 19:42:32 +00:00

historic_bruno commented

2013-08-08 22:33:14 +00:00

Author

Latest plan is to have "our" own autobuild server running Jenkins, as outlined here: JenkinsSetup

The autobuilt files will be stored in an SVN repo and retrieved by the update script according to whatever git branch the user has checked out. It would be nice if all dependencies also had MSVC projects that could be built automatically this way, instead of the current approach of building them individually with a mix of compilers, then forgetting how it's done. But that's not needed initially.

(Apart from git, we want to upgrade the autobuild compiler to VC++ 2010 or 2012, but Philip says it's as much work to do that with the current system as it would be to implement the new one.)

Latest plan is to have "our" own autobuild server running Jenkins, as outlined here: [JenkinsSetup](JenkinsSetup) The autobuilt files will be stored in an SVN repo and retrieved by the update script according to whatever git branch the user has checked out. It would be nice if all dependencies also had MSVC projects that could be built automatically this way, instead of the current approach of building them individually with a mix of compilers, then forgetting how it's done. But that's not needed initially. (Apart from git, we want to upgrade the autobuild compiler to VC++ 2010 or 2012, but Philip says it's as much work to do that with the current system as it would be to implement the new one.)

k776 modified the milestone from Alpha 14 to Alpha 15

2013-08-15 20:19:15 +00:00

leper modified the milestone from Alpha 15 to Alpha 16

2013-12-07 15:03:47 +00:00

leper modified the milestone from Alpha 16 to Alpha 17

2014-04-12 21:01:26 +00:00

Ykkrosh was unassigned by historic_bruno

2014-05-19 02:23:04 +00:00

leper modified the milestone from Alpha 17 to Alpha 18

2014-08-24 11:38:41 +00:00

Itms modified the milestone from Alpha 18 to Alpha 19

2015-02-04 14:47:54 +00:00

mimo added

and removed

1: Release Blocker

labels 2015-07-15 17:34:09 +00:00

mimo modified the milestone from Alpha 19 to Alpha 20

2015-07-15 17:34:09 +00:00

mimo commented

2016-01-28 19:14:11 +00:00

moved to backlog as no progress since ages

moved to backlog as no progress since ages

mimo modified the milestone from Alpha 20 to Backlog

2016-01-28 19:17:43 +00:00

viky commented

2018-06-27 14:56:53 +00:00

Patch removed.

**Patch removed.**

Sign in to join this conversation.

No Label

1: Release Blocker

4: Nice To Have

5: If Time Permits

Art & Animation

Build & Packages

Internationalization & Localization

Multiplayer Lobby

Music & Sound FX

Non-game systems

UI – Game setup

UI – Miscellaneous

UI & Simulation

No Milestone

No project

No Assignees

6 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: wfg/0ad#1819

No description provided.