506404 - Create buildbot steps and automation branch for "nanojit-central"

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Graydon Hoare :graydon]

Hi,

The JS and Tamarin teams (Mozilla and Adobe) are in the process of promoting nanojit to a top-level project with a single repository, its own makefile and testsuite.

I've got IT to put together a new repository, http://hg.mozilla.org/nanojit-central, which is where we'll be putting the code. They said I should open a RelEng bug for getting tinderboxes put on it.

What I'd be looking for initially are tinderboxes running on arm-linux, x86-osx, x86-linux and x86-winnt, with more to come in the future (probably x64-linux and arm-wince, maybe x64-osx or x64-winnt as well)

There's no code in that repository yet, but I'd like to have all this working soon-ish as infrastructure since we're still working on the merge, and having good test coverage during the merge will help a lot.

For the time being, simply pulling and doing a "make clean; make; make check" triple is fine. The code and testsuite will be (compared to firefox) absolutely tiny and doing a full clobber/build/test cycle should only take a minute or so.

Thanks.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Please answer questions about systems/toolchains/etc from here: https://wiki.mozilla.org/ReleaseEngineering:ProjectBranchPlanning into this bug. If it helps, for prior examples, see bug#459269 and bug#500755.

Also, for the sake of hg.m.o hygiene, would you mind if we move this new empty repo to http://hg.mozilla.org/projects/nanojit-central along with the other project branch repos?

Comment 2

[Graydon Hoare :graydon]

Awesome. Sorry for not having found that page earlier.

For reference, I've already filed a releng bug 506404 for the tinderboxes.

Moving it to projects/nanojit-central is fine by me. I'll fill out the form questions presently.

Comment 3

[Graydon Hoare :graydon]

Er, oops, got this bug confused with the IT one and thought I was commenting on it. Long day. Anyway, here's the stock description:

* do you want builds?
  Yes
o which o.s.?
  linux-arm, linux-x86, linux-x64, win32-x86, wince-arm, winnt-x64, osx-x86. as many platforms as possible. Cycle time will be very short, broad coverage would be great. There are *exceptionally* good odds of any nanojit change breaking one CPU/os combination, while passing on others. That is, unfortunately, why I'm asking for all this :(
o incremental-build-on-checkin?
  Yes
o nightlies?
  No
o Are en-US builds enough, with no l10n?
  Yes, en-US only is fine
* do you want unittests?
  Yes
o which o.s.?
  All of them
* do you want talos?
  Some day in the future, probably; but I don't think it's relevant at this point since talos scripts have no idea how to measure plain nanojit performance.
* name of branch owner, who will:
  Graydon, I suppose. Or Edwsmith at Adobe.
* timeline:
o when can we start project branch
  The branch is a completely distinct history from m-c, different contents. Can start immediately. Currently the repository is empty.
o approx expected life span of project branch - if known? 
  Perpetual lifespan, we'll be transplanting changes regularly to other branches.
o need any changes to toolchain used in m-c?
  msvc and gcc, should be fine as-is.
o need any changes to the compile/link/repack steps used in m-c?
  Yes. Custom makefile will appear, unrelated to any others in any mozilla tree. Can tailor makefile target names etc. to your needs.
o preference on tinderbox waterfall name?
  nanojit
o preference on where to put builds on ftp.m.o?
  N/A, archived builds not required
o preference on name of project branch in hg?
  nanojit-central
o what unofficial projectname do you want on this project branch?
  nanojit

Comment 4

[[e-mail] (:bhearsum)]

(In reply to comment #3)
> Er, oops, got this bug confused with the IT one and thought I was commenting on
> it. Long day. Anyway, here's the stock description:
> 
> * do you want builds?
>   Yes
> o which o.s.?
>   linux-arm, linux-x86, linux-x64, win32-x86, wince-arm, winnt-x64, osx-x86. as

Windows x-64 isn't possible at this time - we have exactly zero machines running it.

As this is a non-Mozilla based project I assume the build process is different (eg, no mozconfig, client.mk, different Makefile, targets, etc.). If that's true, we'll need the proper build process recorded somewhere.

Comment 5

[Graydon Hoare :graydon]

(In reply to comment #4)

> Windows x-64 isn't possible at this time - we have exactly zero machines
> running it.

No worries. Thought I'd mention every possibility in case it exists.

> As this is a non-Mozilla based project I assume the build process is different
> (eg, no mozconfig, client.mk, different Makefile, targets, etc.). If that's
> true, we'll need the proper build process recorded somewhere.

Correct. There's no Makefile yet so I can't say for sure -- we're in the process of untangling the code from Adobe's and Mozilla's respective build infrastructures -- but I figured something simple and old-fashioned like:

  rm -Rf $builddir
  mkdir $builddir
  cd $builddir
  $srcdir/configure
  make
  make check

We should be able to adapt whatever we're doing to that.

Comment 6

[Graydon Hoare :graydon]

We'll have the makefile for this committed soon-ish -- next week or so -- would be good to consider actually getting tinderboxes online here.

Comment 7

[Graydon Hoare :graydon]

I've brought the tinderboxes online myself, and tested that at MozillaTest tree. Would it be possible to get a new tree-display called "Nanojit" established, that I can send the results to instead of MozillaTest? This will be a permanent tree.

I'm also having a little difficulty getting the "TinderboxPrint:" scraping stuff to work. Reading the tinderbox source, it looks a bit like it needs manual configuration on the server, to list a particular set of build names?

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

http://tinderbox.mozilla.org/showbuilds.cgi?tree=Nanojit has been created. Once machines start reporting there then we (actually, anyone with the tbox server pw which includes lots of sheriffs) can enable scraping to pick up the TinderboxPrint's.

Comment 9

[Graydon Hoare :graydon]

Now sending data to that tree, any time you want to turn on scraping, I'd appreciate it.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Done for the four machines reporting currently. Builds will be scraped from now on.

Comment 11

[Jordan Gutterman]

I'm curious why this is the only *-central repo in hg.mozilla.org/projects/ rather at the top level. The other stuff in that dir seem to be mostly things not meant to be as productized as nanojit is. (Although looking at what's in there it doesn't seem to follow much of a pattern: electrolysis I'm guessing only will exist until it gets merged into trunk, places doesn't seem to get random temp landings, and 2007-configure-rewrite seems quite dead, htmlparser seems to be the only permanent thing).

Comment 12

[Graydon Hoare :graydon]

I don't care in the least whether it's in /projects or not. I just figured it'd be decorous to move it out of /users/graydon_mozilla.com.

Comment 13

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

I would approve of a move to hg.m.o/nanojit-central, for what little it's worth.

Comment 14

[Jordan Gutterman]

(In reply to comment #13)
> I would approve of a move to hg.m.o/nanojit-central, for what little it's
> worth.

Ok, I filed bug 522412.

Comment 15

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #11)
> I'm curious why this is the only *-central repo in hg.mozilla.org/projects/
> rather at the top level. The other stuff in that dir seem to be mostly things
> not meant to be as productized as nanojit is. 
Per comment#1, comment#2, this repo was created in projects to reduce top-level clutter. If this location is no longer valid, or if the business requirements for this project have changed, we can move the repo as you wish, but it would be great if you could help me understand why the current location is an issue??

> (Although looking at what's in
> there it doesn't seem to follow much of a pattern: electrolysis I'm guessing
> only will exist until it gets merged into trunk, places doesn't seem to get
> random temp landings, and 2007-configure-rewrite seems quite dead, htmlparser
> seems to be the only permanent thing).
IT and RelEng are (slowly) cleaning the directory structure in hg.m.o. For example, unrelated to nanojit, we also have some obsolete repos at the top-level, which should be moved to help declutter that top-level directory.

Comment 16

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

What's the business requirement behind wanting fewer top-level directories?  Flatter is better than deeper for us, generally.

Comment 17

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #16)
> What's the business requirement behind wanting fewer top-level directories? 
> Flatter is better than deeper for us, generally.

How these are organized is obviously a style thing. One level of directories seemed a good balance between a nested tree, and just cluttering everything into the same toplevel. But as I said, its just one style of approach to the problem of having many many repos on hg.m.o.


(In reply to comment #15)
> (In reply to comment #11)
> Per comment#1, comment#2, this repo was created in projects to reduce top-level
> clutter. If this location is no longer valid, or if the business requirements
> for this project have changed, we can move the repo as you wish, but it would
> be great if you could help me understand why the current location is an issue??

Again, if you want this repo moved from "projects" to top-level we can happily do that. 

I'm just trying to understand whats changed since we originally created it, so I know what to look for when working on other new repo requests?

Comment 18

[Jordan Gutterman]

(In reply to comment #15)
> Per comment#1, comment#2, this repo was created in projects to reduce top-level
> clutter. If this location is no longer valid, or if the business requirements
> for this project have changed, we can move the repo as you wish, but it would
> be great if you could help me understand why the current location is an issue??
>
> IT and RelEng are (slowly) cleaning the directory structure in hg.m.o. For
> example, unrelated to nanojit, we also have some obsolete repos at the
> top-level, which should be moved to help declutter that top-level directory.

Well, I don't know about the business requirements of the project. I guess what see is that Nanojit is most closely related to Tamarin and Tracemonkey, and there are 6 repos that use it (Tamarin-*, Tracemonkey, Actionmonkey-*) that are all currently at the top level, so that's the easiest directory to find this one. It kinda seems to me like they belong in the same place. Cleaning and decluttering is good, but moving all those would probably be somewhat disruptive.

Comment 19

[Brendan Eich [:brendan]]

There's nothing "cleaner" about a projects directory to be used inconsistently, make longer path parts, and hike keystroke taxes. From the Zen of Python (some of which I find un-Zen, or uncool, but most is good):

...
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
...

/be

Comment 20

[Chris Cooper [:coop] (he/him)]

Re-assigning to joduinn at his request so he can pin down the priority on this.

Comment 21

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #12)
> I don't care in the least whether it's in /projects or not. I just figured it'd
> be decorous to move it out of /users/graydon_mozilla.com.

As Graydon is the branch owner, and likes the repo where it is, so be it. 

Graydon, there was some eaerly discussion about who needed access to this repo. I see commits happening, so is this all sorted out now, or is there anything left to sort out about permissions?


Moving over to Lukas, as she's winding up a project, and this will be next on her list.

Comment 22

[Lukas Blakk [:lsblakk] use ?needinfo]

Ready to start generating patches to try this in staging.  From what was mentioned in this bug, it requires custom steps - what steps should we be running on these builds?

Comment 23

[Nicholas Nethercote [inactive]]

Commits to this repo are working well.  It'd be great to have centralized tinderbox machines running though, as Graydon's home-brew setup is a little unreliable.

The home-brew setup is controlled by the scripts at
http://hg.mozilla.org/users/graydon_mozilla.com/nanojit-tinderbox.  I don't know anything about tinderbox setup but hopefully those scripts will be helpful for someone more in the know?

One good thing about Nanojit is that it's really small -- on my X64/Linux box it takes about 5s to build and about 10s to run the tests.

Keep asking if I haven't answered your questions...

Comment 24

[Graydon Hoare :graydon]

There is a small amount of mail-server configuration (self-explanatory) in the first few lines of the script. Aside from that, for the most part all one needs to do is run ./nanojit-tinderbox.py --target=foo, in some writable workspace. It'll poll, update, configure, build and test in a loop. I run copies of it with --target= set to each of:

i386-unknown-darwin
x86_64-unknown-darwin
linux-armv7l
i686-pc-mingw
i686-unknown-linux-gnu
x86_64-unknown-linux-gnu

Notably, however: it is *not* in any way integrated into the normal job scheduling system inside releng. It may therefore be quite useless to your integration task.

Again, it's important to differentiate these from the "normal" firefox builds when trying to figure out how to integrate. Nanojit is a standalone program held in its own (very small) repository with no direct relationship to a firefox build. It's not just a "firefox variant". There's no browser, none of the normal browser config/build/test logic applies. It's a library + a small test-driver program.

Comment 25

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Found bug#522412 during cleanup; adding here for completeness.

Comment 26

[Lukas Blakk [:lsblakk] use ?needinfo]

Taking myself off this bug for now since I'm not actively working on it.

Comment 27

[Lukas Blakk [:lsblakk] use ?needinfo]

Graydon - I'm taking this bug back but wondering what the status is now - do you still want an entire project branch or could the disposable branches do what you need?

Comment 28

[Robert Sayre]

we're keeping this repository up permanently.

Comment 29

[Graydon Hoare :graydon]

Yeah. It's a forever thing. I'm still running the tinderboxes on my desktop, fwiw.

To remind / reiterate comment 24: this is not a firefox or other mozilla-tree variant. Nanojit isn't a browser build. It's an independent software project with an independent build-and-test system. Can't be considered a "branch" of mozilla's tree. Nanojit gets transplanted into a subdirectory of the main mozilla tree. Think of this like "sqlite that we help maintain" or something. It's an individual library, but every mozilla product (thunderbird, firefox, fennec, etc. etc.) depends on it. Isn't going away any time soon.

Comment 30

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #21)
> (In reply to comment #12)
> Graydon, there was some eaerly discussion about who needed access to this repo.
> I see commits happening, so is this all sorted out now, or is there anything
> left to sort out about permissions?

Are there users of this repo who are not already able to checkin to mozilla-central? (I ask because this impacts what slaves we should run these builds/tests on).



(In reply to comment #29)
> Yeah. It's a forever thing. I'm still running the tinderboxes on my desktop,
> fwiw.
> 
> To remind / reiterate comment 24: this is not a firefox or other mozilla-tree
> variant. Nanojit isn't a browser build. It's an independent software project
> with an independent build-and-test system. Can't be considered a "branch" of
> mozilla's tree. 
Good to know. See questions below.


> Nanojit gets transplanted into a subdirectory of the main
> mozilla tree. Think of this like "sqlite that we help maintain" or something.
> It's an individual library, but every mozilla product (thunderbird, firefox,
> fennec, etc. etc.) depends on it. Isn't going away any time soon.
If nanojit is being transplanted into Firefox tree anyway, have you considered using the nanojit-in-firefox for building and testing? We could "easily" set this up as a project branch with our existing infrastructure. Are there nanojit-specific build-types, or tests that you are looking for? Are there nanojit-specific standalone distributions being built, or is it enough to just build as-part-of Firefox?

Comment 31

[Nicholas Nethercote [inactive]]

(In reply to comment #30)
> 
> Are there users of this repo who are not already able to checkin to
> mozilla-central? (I ask because this impacts what slaves we should run these
> builds/tests on).

I'm not sure, but I suspect the Adobe guys (eg. edwsmith) don't have m-c access.

> > Nanojit gets transplanted into a subdirectory of the main
> > mozilla tree. Think of this like "sqlite that we help maintain" or something.
> > It's an individual library, but every mozilla product (thunderbird, firefox,
> > fennec, etc. etc.) depends on it. Isn't going away any time soon.
>
> If nanojit is being transplanted into Firefox tree anyway, have you considered
> using the nanojit-in-firefox for building and testing? We could "easily" set
> this up as a project branch with our existing infrastructure. Are there
> nanojit-specific build-types, or tests that you are looking for? Are there
> nanojit-specific standalone distributions being built, or is it enough to just
> build as-part-of Firefox?

Nanojit isn't distributed by itself, only as part of TraceMonkey (Mozilla) and Tamarin (Adobe).  There aren't nanojit-specific build-types, AFAIK.  If we tested nanojit-in-firefox would the overheads increase?  Currently nanojit-central takes only a minute or two to run all its tests, this is a feature I'd really like to keep if possible.

Comment 32

[Edwin Smith]

(In reply to comment #31)
> I'm not sure, but I suspect the Adobe guys (eg. edwsmith) don't have m-c
> access.

I believe I was recently upgraded to be an L3 committer, but I have never commited to m-c.  A subset of the tamarin team needs to commit to nanojit-central but nobody is a mozilla-central committer.  I'm being vague because I'm not sure how granular commit access is.

Comment 33

[Graydon Hoare :graydon]

L3 is not m-c commit rights.

I'm surprised at the direction this bug is taking. Are we so infrastructurally wedded to only building firefox-tree variants that we want to corral all other projects we develop into this format? This seems like a problem. Nanojit is only a handful of files; it probably takes longer to *check out* a m-c tree than perform the entire nanojit build/test cycle.

(I ask at least partly because, somewhat off topic, I'm also running the tinderboxes for Rust on the same pile of desktop systems doing the nanojit tinderboxes, and those are also "not a firefox tree". Surely mozilla contributes to quite a number of not-a-firefox-tree projects; I guess I'm asking if we are, by policy, to be "on our own" when it comes to build/test farms.)

Comment 34

[Chris AtLee [:catlee]]

No, there's no reason why can't do non-firefox builds.  I've got some ideas how to move forward here.

Comment 35

[Lukas Blakk [:lsblakk] use ?needinfo]

> I'm surprised at the direction this bug is taking. Are we so infrastructurally
> wedded to only building firefox-tree variants that we want to corral all other
> projects we develop into this format? 

The purpose of the question was to clarify so we could change the summary of the bug and look into where it could be properly place for best results.  It was originally assigned to me because I was taking care of adding new project branches (mozilla-central based branches) but since this is its own code and steps this is a different kind of work and we want to be able to classify it accordingly and assign to the right person.  Thanks for the replies, all.

Comment 36

[Brendan Eich [:brendan]]

Nanojit people from Adobe and elsewhere should not have to get L3 access. L3 is combining with L2, last I heard.

Also, what Graydon said -- but I should not spout off, because it sounds like everything is cool. I did want to emphasize that one size does not fit all. We need more small, well-tested, and well-factored repos like Nanojit. Thanks,

/be

Comment 37

[Chris AtLee [:catlee]]

> * do you want unittests?
>   Yes

Which unittests are these?  How do we run them?

Comment 38

[Nicholas Nethercote [inactive]]

(In reply to comment #37)
> > * do you want unittests?
> >   Yes
> 
> Which unittests are these?  How do we run them?

Just do 'make check' in a build directory.  You should see output like that below.  It takes less than 15 seconds to run them on my Mac laptop.


[wave:~/moz/nanojit-central/debug32] make check
../lirasm/testlirc.sh bin/lirasm
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/add.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/addjovi.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/addjovi_ovf.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/addsub.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/call1.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/call2.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/f2i.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/floatingpoint.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/fuzz-527178.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/loadstore.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mul_xxx.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mul_xxy.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mul_xyy.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mul_xyz.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/muljovi.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/muljovi_ovf.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mulov_xxx.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mulov_xxy.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mulov_xyy.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/mulov_xyz.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/multfrag1.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/multfrag2.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/multfrag3.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/std2f.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/subjovi.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/subjovi_ovf.in
TEST-PASS | lirasm | lirasm  --execute --random 1000000
TEST-PASS | lirasm | lirasm  --execute --random 1000000 --optimize
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/64-bit/dasq.in
TEST-PASS | lirasm | lirasm  --execute ../lirasm/tests/64-bit/qasd.in

Comment 39

[Chris AtLee [:catlee]]

Can you take a look at this output to see everything looks ok:

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1283372908.1283373123.31897.gz&fulltext=1

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1283373017.1283373053.31518.gz

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1283373018.1283373055.31521.gz

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1283372873.1283372914.30818.gz

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1283372863.1283372939.30938.gz

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTest/1283373279.1283373318.32661.gz

Comment 40

[Nicholas Nethercote [inactive]]

(In reply to comment #39)
> Can you take a look at this output to see everything looks ok:

It does!  Those failures on ARM (the first link) are expected.  This is very promising, thanks for looking into it :)

Comment 41

[Chris AtLee [:catlee]]

Attachment: nanojit script and hgtool

Comment 42

[Chris AtLee [:catlee]]

Attachment: project configs for nanojit

Comment 43

[Chris AtLee [:catlee]]

Attachment: generateNanojitObjects for creating nanojit hg poller, build factories, etc.

Comment 44

[[e-mail] (:bhearsum)]

Comment on attachment 474024 [details] [diff] [review]
nanojit script and hgtool

runCmd/getOutput seem like something more general purpose. Back in the Perl days we had http://mxr.mozilla.org/mozilla/source/tools/release/MozBuild/Util.pm#49, it would be nice to start on something like that for Python. Can you move these to lib/python somewhere?


>+def doClone(repo, dest, branch=None, revision=None):
>+    if os.path.exists(dest):
>+        removePath(dest)
>+
>+    cmd = ['hg', 'clone']
>+    cmd.extend([repo, dest])
>+    runCmd(cmd)
>+
>+    # Check & switch branch
>+    local_branch = getOutput(['hg', 'branch'], cwd=dest).strip()
>+    # If this is different, checkout the other branch
>+    if branch != local_branch:
>+        runCmd(['hg', 'checkout', branch], cwd=dest)

Why 'checkout' here and 'update' down there? And why not -C up here?


>+
>+    # Update
>+    if revision is not None:
>+        cmd = ['hg', 'update', '-C', '-r', revision]
>+        runCmd(cmd, cwd=dest)
>+
>+    return getRevision(dest)
>+
>+def doUpdate(repo, dest, branch=None, revision=None):
>+    """Clone the hg repo at `repo` into destination directory `dest`
>+    After cloning, update to `branch` and `revision`, if set.

Does this docstring belong to doClone?


>+
>+    Otherwise update to the default branch."""
>+    paths = getOutput(['hg', 'paths'], cwd=dest)
>+    if repo not in paths:
>+        raise ValueError("Looks like dest isn't related to %s" % repo)
>+
>+    # Pull
>+    cmd = ['hg', 'pull', '-f']

Why -f if you're erroring out if the repositories are unrelated? Is there something specific you're forcing?

>+    if revision is not None:
>+        cmd.extend(['-r', revision])
>+    cmd.append(repo)
>+    runCmd(cmd, cwd=dest)
>+
>+    # Check & switch branch
>+    local_branch = getOutput(['hg', 'branch'], cwd=dest).strip()
>+    # If this is different, checkout the other branch
>+    if branch != local_branch:
>+        runCmd(['hg', 'checkout', branch], cwd=dest)

Same thing here re: checkout/-C


All of the logic seems fine.

Comment 45

[[e-mail] (:bhearsum)]

Comment on attachment 474026 [details] [diff] [review]
generateNanojitObjects for creating nanojit hg poller, build factories, etc.

Looks good to me

Comment 46

[Chris AtLee [:catlee]]

Comment on attachment 474024 [details] [diff] [review]
nanojit script and hgtool

I'll fix up hgtool to address the comments.

Comment 47

[Armen [:armenzg]]

I am little late but I hope any of my feedback wrt to hgtool can helpout.

* I love how this is looking like
* Could you please have a brief description on the purpose of hgtool.py?
* How do we handle shutil.rmtree() exceptions? (you have it covered by checking for existance before calling os.path.exists but I worry about the future when we grow the code)
* How do we handle CalledProcessError exceptions in subprocess.check_call? I guess we would just raise all the way up and the build would turn purple and see the exception on the buildbot log?
* On getOutput you have "include_stderr"; where does stderr go to when not set to True? Would we see it on the logs?
* What happens with p.wait() if the command never finishes? 
* What is the difference between runCmd and getOutput? Why in one of them you use check_call and the other Popen? It seems that check_call returns the output *and* the return code while on getOutput only the output is returned (the retcode is not on the output)
* if any of the runCmd calls on doClone fail how do we handle it? Do we want to retry?
* after we get our revision (by the end of calling __main__) is it just good enough to finish without returning 0? It seems that we either end or throw an exception (please forgive me if this is a newbies question - the others might be almost newbie too)

Comment 48

[Armen [:armenzg]]

Comment on attachment 474025 [details] [diff] [review]
project configs for nanojit

Looks fine.

Comment 49

[Armen [:armenzg]]

Comment on attachment 474026 [details] [diff] [review]
generateNanojitObjects for creating nanojit hg poller, build factories, etc.

Stamp.

Comment 50

[Chris AtLee [:catlee]]

Attachment: Refactored scripts

Comment 51

[[e-mail] (:bhearsum)]

Comment on attachment 476270 [details] [diff] [review]
Refactored scripts

Looks good!

Comment 52

[Armen [:armenzg]]

Comment on attachment 476270 [details] [diff] [review]
Refactored scripts

I don't want to block you on my review. Ben's review should be good enough.

Comment 53

[Chris AtLee [:catlee]]

Comment on attachment 474025 [details] [diff] [review]
project configs for nanojit

changeset:   3026:3c440954c0ed

Comment 54

[Chris AtLee [:catlee]]

Comment on attachment 474026 [details] [diff] [review]
generateNanojitObjects for creating nanojit hg poller, build factories, etc.

changeset:   963:90f79eee7b4e

Comment 55

[Chris AtLee [:catlee]]

Comment on attachment 476270 [details] [diff] [review]
Refactored scripts

changeset:   810:771ba95aa74f

Comment 56

[Chris AtLee [:catlee]]

Should be working now.  Please re-open or file new bugs if you see anything amiss.

Comment 57

[Graydon Hoare :graydon]

Awesome. Shall I take my desktop tinderboxes offline? The arm one is still ... "in recovery" and unlikely to actually get back to "working" this week.

Comment 58

[Nicholas Nethercote [inactive]]

Thanks for getting this going!

Am I right to think that http://tinderbox.mozilla.org/showbuilds.cgi?tree=Nanojit now shows the central NJ tests rather than Graydon's ones?

There's a problem, though -- there are test failures (TEST-UNEXPECTED-FAILURE) on ARM and Windows (at least) but they're showing up green.

Comment 59

[Chris AtLee [:catlee]]

(In reply to comment #58)
> Thanks for getting this going!
> 
> Am I right to think that
> http://tinderbox.mozilla.org/showbuilds.cgi?tree=Nanojit now shows the central
> NJ tests rather than Graydon's ones?

Not sure what the distinction is here.

> There's a problem, though -- there are test failures (TEST-UNEXPECTED-FAILURE)
> on ARM and Windows (at least) but they're showing up green.

Looks like maybe 'make check' is exiting with a 0 error code even when there are test failures?

Comment 60

[Graydon Hoare :graydon]

(In reply to comment #59)
> (In reply to comment #58)
> > Thanks for getting this going!
> > 
> > Am I right to think that
> > http://tinderbox.mozilla.org/showbuilds.cgi?tree=Nanojit now shows the central
> > NJ tests rather than Graydon's ones?
> 
> Not sure what the distinction is here.

My tinderboxes -- the ones running on computers sitting on my desk in front of me -- are still reporting, at least non-ARM. The ARM one is offline. I can take them all offline now, if it's likely to confuse matters.

Comment 61

[Nicholas Nethercote [inactive]]

(In reply to comment #59)
> 
> > There's a problem, though -- there are test failures (TEST-UNEXPECTED-FAILURE)
> > on ARM and Windows (at least) but they're showing up green.
> 
> Looks like maybe 'make check' is exiting with a 0 error code even when there
> are test failures?

Ah, yes, thanks for pointing that out.  I fixed that, but now boxes with test failures (nanojit-arm, nanojit-win32) are showing as red instead of orange.  Any idea how to fix that?

Also, in the columns for the releng machines, the revhashes don't link to the diff, unlike the columns for Graydon's machines.  Could that be changed?  (Actually, nanojit-arm and nanojit-linux64 don't even show the revhashes.)

Finally, I just talked with Graydon and he's turned off the reporting from his machines, so those columns should disappear soon.

Thanks!

Comment 62

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #61)
> Also, in the columns for the releng machines, the revhashes don't link to the
> diff, unlike the columns for Graydon's machines.  Could that be changed? 

catlee, looks like a missing --tbox when calling hgtool.py.

> (Actually, nanojit-arm and nanojit-linux64 don't even show the revhashes.)

I've set 'scrape' enabled in the waterfall config for the linux64 box, arm already had it.

Comment 63

[Chris AtLee [:catlee]]

(In reply to comment #62)
> (In reply to comment #61)
> > Also, in the columns for the releng machines, the revhashes don't link to the
> > diff, unlike the columns for Graydon's machines.  Could that be changed? 
> 
> catlee, looks like a missing --tbox when calling hgtool.py.

Well, what's really happening is that scratchbox isn't inheriting the full environment, so isn't enabling --tbox by default when the build properties environment variable is set.  Also, we're not printing out links to the revisions, just the raw revision.  Not sure what the right solution is here...Have hgtool spit out HTML if --tbox is enabled?

Per IRC, I'm going to modify the nanojit.sh script as well as ScriptFactory to treat exit code 1 as a warning, and exit code 2 (really not 1,0) as an error.

Comment 64

[Chris AtLee [:catlee]]

Attachment: Fix exit codes, print out links to revision

This changes nanojit.sh to exit with 1 if 'make check' fails, and with 2 if other steps fail.  I'll post a patch to ScriptFactory shortly that do warnings/failures depending on exit code.

I also tweaked hgtool to print out a link to the revision when TinderboxPrint is enabled.

Comment 65

[Chris AtLee [:catlee]]

Attachment: Add support for warnings on certain exit codes to ScriptFactory

Comment 66

[[e-mail] (:bhearsum)]

Comment on attachment 479753 [details] [diff] [review]
Add support for warnings on certain exit codes to ScriptFactory

How do you feel about making this more robust and taking something in the form of:
{WARNINGS: (exit codes),
 FAILURE: (exit codes)
}

etc.

Comment 67

[Chris AtLee [:catlee]]

Attachment: Add support for warnings on certain exit codes to ScriptFactory

Comment 68

[Chris AtLee [:catlee]]

Attachment: Fix exit codes, print out links to revision

rebased, added --tbox to work around scratchbox issue for now.

Comment 69

[[e-mail] (:bhearsum)]

Comment on attachment 480235 [details] [diff] [review]
Fix exit codes, print out links to revision

build/tools landing: 00d892839b05

Comment 70

[[e-mail] (:bhearsum)]

Comment on attachment 479790 [details] [diff] [review]
Add support for warnings on certain exit codes to ScriptFactory

buildbotcustom landing: e9f1f198c07b

Comment 71

[[e-mail] (:bhearsum)]

I'm pretty sure we're all done here now.

Comment 72

[Nicholas Nethercote [inactive]]

I just pushed to nanojit-central.  I'm still seeing red for nanojit-arm and nanojit-win32, but they should be orange, because they have test failures, not compile failures.

The build logs, in case they're of any use:

http://tinderbox.mozilla.org/showlog.cgi?log=Nanojit/1286232809.1286232911.14238.gz
http://tinderbox.mozilla.org/showlog.cgi?log=Nanojit/1286232809.1286233042.14806.gz

Comment 73

[[e-mail] (:bhearsum)]

Catlee will have to look at that when he's back on Tuesday =\

Comment 74

[Chris AtLee [:catlee]]

Attachment: Fix up log/error handling

Recent buildbot upgrades changed the behaviour of the return code handling.  This should fix it!

Comment 75

[Chris AtLee [:catlee]]

Comment on attachment 483447 [details] [diff] [review]
Fix up log/error handling

changeset:   1009:0ddec2679911

Comment 76

[Chris AtLee [:catlee]]

This should be fixed now!

Comment 77

[Nicholas Nethercote [inactive]]

(In reply to comment #76)
> This should be fixed now!

Indeed it is, many thanks!