<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">I was discussing this very problem with Thomas in fact! Right now we deliberately do a separate opam install for each package, and do not cover depopts at all.<div><br></div><div>The plan is to integrate the scripts into using Irmin to record the log outputs, since git is hitting congestion collapse with >20 workers (GitHub rejects the push due to another process having the lock). With Irmin, we can record all the logs into a memory instance, and have one pushing process to GitHub.</div><div><br></div><div>When we do have Irmin support, we can calculate a static schedule of all the packages that should be built, and just have spare CPUs pick up some work. It would be great to build the risky packages (i.e. early up the dependency chain) first, and then move to the leaf packages.</div><div><br></div><div>I'm not sure if re-using packages and snapshotting is a good idea in general, but it's very useful for common ones like ocamlfind, Lwt or Core. Depopts are a big problem here though, particularly for Lwt...</div><div><br></div><div>-anil</div><div><br><div><div>On 23 May 2014, at 09:50, Louis Gesbert <<a href="mailto:louis.gesbert@ocamlpro.com">louis.gesbert@ocamlpro.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: monospace; font-size: 8pt;"><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">Very nice !</div><p style="white-space: pre-wrap; margin: 0px; text-indent: 0px;"> </p><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">I am wondering, do you 'opam install' every package individually ? We may play some clever tricks with the solver here, like `opam upgrade --criteria=-notuptodate,+new` which will attempt to install as many packages as possible, after which we could iterate on a universe reduced to the not-yet-installed packages. Depopts would be less predictable, but we can't realistically test all possible combinations of those anyway.</div><p style="white-space: pre-wrap; margin: 0px; text-indent: 0px;"> </p><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">Would be nice to find a common place to aggregate such results.</div><p style="white-space: pre-wrap; margin: 0px; text-indent: 0px;"> </p><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">Best,</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">Louis</div><p style="white-space: pre-wrap; margin: 0px; text-indent: 0px;"> </p><p style="white-space: pre-wrap; margin: 0px; text-indent: 0px;"> </p><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">Le mardi 20 mai 2014, 17:40:04 Anil Madhavapeddy a écrit :</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> Hi,</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> </div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> I've written some scripts that use Docker and OPAM to run bulk builds across the entire package repository, and the results are logged and pushed to GitHub daily.</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> </div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> If anyone wants to improve the health of the repository, please look at:</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> </div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> <a href="https://github.com/ocaml/opam-bulk-logs">https://github.com/ocaml/opam-bulk-logs</a> </div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> </div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> in the err/ directories, and submit pull requests fix the failures.</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> </div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> Source scripts are still in my personal GitHub; I'll move these to the ocaml/ repo.</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> </div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> cheers,</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> Anil</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> _______________________________________________</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> opam-devel mailing list</div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> <a href="mailto:opam-devel@lists.ocaml.org">opam-devel@lists.ocaml.org</a></div><div style="white-space: pre-wrap; margin: 0px; text-indent: 0px;">> <a href="http://lists.ocaml.org/listinfo/opam-devel">http://lists.ocaml.org/listinfo/opam-devel</a></div><p style="white-space: pre-wrap; margin: 0px; text-indent: 0px;"> </p>_______________________________________________<br>MirageOS-devel mailing list<br><a href="mailto:MirageOS-devel@lists.xenproject.org">MirageOS-devel@lists.xenproject.org</a><br><a href="http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel">http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel</a></div></blockquote></div><br></div></body></html>