[ocaml-platform] Dev Version as Package?

[Trevor Smith]

Louis,

Thanks for your responses. Apologies for my late response. I only just got
time to try this all out.

I tried out the workarounds but am not seeing it pick up new commits made
to a git repo.

Example (not using dependencies):

1) Create package "foo" in a custom repo "my-repo". With url file set to a
git repo.
2) Add repo.
3) opam install foo
4) Make new commit and push to foo's repo
<following your suggested workflow>
5) opam update -u my-repo
<this returns Everything as up-to-date as possible.>
6) opam install foo
<package is already installed>

I think I am missing something from your suggestions -- perhaps this is
clearly not going to work because OPAM does not ever check the upstream git
repo?). Was that workflow meant to only work with a pinned version?

Thanks.

Trevor

On Sun, May 10, 2015 at 11:13 PM, Louis Gesbert <louis.gesbert at ocamlpro.com>
wrote:

> > 1) I think that `opam update -u <my-internal-repo>` and then `opam
> upgrade`
> > would indeed do what I need as a workaround.
>
> `update -u` is "update then upgrade" so no need for the upgrade afterwards
> :)
>
> > The incidental complexity the it creates isn't that big of a deal for
> > one-offs on my box, but when managing a team of devs, and build boxes one
> > wants something straightforward.
> > It seems though that to make that work with the build system and to not
> > have unintended updates one would then need 2 internal dev repos -- one
> > that had all of the git versioned ones, and another repo that had your
> > normal x.y.z versions.
>
> Not sure I get what you are after exactly, but you could also put both
> version in the same repository! Making separate repositories would be
> useful if you want to easily configure hosts for the dev or release
> versions.
>
> > 2) I like my suggested solution better because I think it adds no
> > incidental complexity -- one has to describe what needs describing (ie "I
> > want this dependency always updated"), and no extra lifecycle commands
> (ie
> > no explicit "update" after one has already pushed code). And adds a clear
> > functionality to the tool.
> >
> > Agreed that updating a ton of packages could take a while -- but having a
> > clear tool (as in what I am suggesting) to define that gives the user
> that
> > flexibility -- it's my fault if I've decided to have 100 "alpha"
> packages.
> > I've asked for it.
> >
> > So I'm clear -- are you saying that due to how updates are currently
> dealt
> > with within opam that the feature I'm suggesting probably would not
> happen?
> > (I haven't taken a look at how opam works).
>
> Granted, there is already the mentionned-pinned-package-auto-update trick,
> but it's quite limited in scope -- I think it's good design otherwise to
> keep matters separate and provide atomic commands: the update of the
> internal metadata, and the installation of packages, are clearly distinct
> operations in OPAM. Higher-level operations can easily be built on top of
> it, and possibly added to OPAM, but we shouldn't put too much magic in the
> base operations: that makes things more difficult to handle, to understand
> and to debug.
>
> There is also a technical issue with what you propose: solving an
> installation is based on the current -- static -- metadata. So that
> metadata can't depend on updating other metadata, or resolving an
> installation would become a nightmare. That doesn't apply, though, if you
> only update the package source and not its metadata (like would be done
> with a repository package pointing to a VC repository, as opposed to a
> pinned package) -- it would be much easier to handle in that case, although
> it could become a bit inconsistent.
>
> With the fixes I mentionned earlier, a possibility would be to add a
> `--update-dev` option to `install` and `upgrade`, meaning something like:
> `opam install --update-dev foo`: get all installed development packages
> transitively related to `foo` (dependencies or dependent), update them
> first, then proceed with the installation, recompiling changed ones.
>
> For `install`, this would be more or less similar to `opam install foo
> $(opam list --rec --required-by foo)`, once we implement the changes to do
> related recompilations on install.
>
> Does that make sense ?
>
> Best,
> Louis
>
> >
> > Thanks.
> >
> > Trevor
> >
> > On Sun, May 10, 2015 at 9:07 PM, Louis Gesbert <
> louis.gesbert at ocamlpro.com>
> > wrote:
> >
> > > OPAM normally never updates its metadata outside of `opam update`; this
> > > was inconvenient for pins, so a trick was added: when a pinned package
> is
> > > mentionned explicitely on an install command-line, it is first updated
> from
> > > its upstream. Updating all the time could get very slow if you have
> many
> > > packages pinned to their upstream, for example (like I do).
> > >
> > > As for reinstallations, OPAM does the minimum reinstallations to
> guarantee
> > > consistency, i.e. it recompiles only packages that depend on a changed
> one
> > > -- not the other way around. It will, however, mark packages that were
> > > changed after an update for reinstallation. This should work in a
> similar
> > > way for repository and pinned packages. However, that reinstallation
> will
> > > only take place on an `opam upgrade` without argument; it would
> probably be
> > > an improvement to process it as soon as it belongs to the dependency
> cone
> > > of changed packages.
> > >
> > > Now for the dirty (and not really working) workarounds:
> > > 1. `alias opam="opam update --dev; opam"`. (this would work with the
> fix
> > > above; and the option --dev doesn't exist at the moment, we only have
> an
> > > option --repositories. Adding it)
> > > 2. Instead of `opam install foo`, fun `opam install $(opam list -s
> --rec
> > > --required-by foo) foo`. This should trigger updates due to the trick
> > > above. The reinstallations are only done on `opam upgrade` though.
> > >
> > > So indeed, your only real option if you want to keep your stuff
> up-to-date
> > > is to run `opam update -u` beforehand (possibly with your dev-repo
> name, or
> > > --dev, depending on whether you use a custom repo or pinned packages).
> > >
> > > Now, if we were to handle planned recompilations more often than only
> on
> > > global `opam upgrade`, would that be enough to handle your use-case
> better ?
> > >
> > > Best,
> > > Louis
> > >
> > > > - Trevor Smith, 10/05/2015 19:55 -
> > > > Thanks Thomas.
> > > >
> > > > Would there be interest in making a feature to "always update this
> > > > dependency"? Suggested solution:
> > > >
> > > > A keyword in the version could trigger "always update". This would
> then
> > > > apply to both normal packages as well as pins. "Dev" already has
> meaning
> > > in
> > > > opam, perhaps "-alpha". This would then allow me to internally have a
> > > > single repo with normal versions alongside versions that are actively
> > > being
> > > > developed and changing multiple times per day. Eg:
> > > >
> > > > my-repo/my-package.1.0-alpha/url:
> > > > src: "ssh://somewhere/my-package.git"
> > > >
> > > > A dependent package (ie reverse dependency) "program" on:
> "my-package" {=
> > > > "1.0-alpha" } would then always check for updates and upgrade
> my-package
> > > > before doing anything with program. A build server could get the
> behavior
> > > > I'm looking for by "opam install --deps-only program" and getting
> > > whatever
> > > > is at head of my-package (or of the given branch as the case may be).
> > > >
> > > > I think this solution would be a nice composition of the ideas that
> opam
> > > > already has, and, I think, would be a strict improvement over the
> current
> > > > situation and does not introduce any incidental complexity. This
> thread
> > > > seems to indicate that others are interested in such a feature.
> > > >
> > > > Thoughts? Thanks!
> > > >
> > > > Trevor
> > > >
> > > >
> > > >
> > > > On Sun, May 10, 2015 at 5:50 PM, Thomas Gazagnaire <
> > > thomas at gazagnaire.org>
> > > > wrote:
> > > >
> > > > > Consider two packages: lib and program. program depends upon lib.
> > > > >
> > > > >
> > > > > opam install lib
> > > > > # Make a new commit to lib and push it
> > > > > # Lib is now one commit newer
> > > > > # NB I am making _no_ changes to the internal opam repo
> > > > > opam install program
> > > > >
> > > > > lib does _not_ get recompiled.
> > > > >
> > > > > Is that the information you wanted?
> > > > >
> > > > >
> > > > > yes, when pin/dev packages are modified, you need to tell opam
> that you
> > > > > want to use the updated version (if available), so you need to run
> > > `opam
> > > > > update -u` before `opam install program`. Opam will check if there
> are
> > > new
> > > > > commits and recompile what needs to be recompiled.
> > > > >
> > > > > Thomas
> > > > >
> > > > >
> > > > > Also: I just tried opam update program and that also did not pick
> up
> > > the
> > > > > fact that lib is git pinned.
> > > > >
> > > > > Thoughts? Thanks.
> > > > >
> > > > > Trevor
> > > > >
> > > > > On Sun, May 10, 2015 at 1:17 PM, Thomas Gazagnaire <
> > > thomas at gazagnaire.org>
> > > > > wrote:
> > > > >
> > > > >> I tried setting up the "option 1" -- using a repo url. This works
> fine
> > > > >> for clean installs, however it does not work for my use case for
> > > updates. I
> > > > >> didn't realize until I tried it out that this won't update upon
> any
> > > > >> dependent installation. As noted by Louis, pinning also does not
> > > reinstall
> > > > >> from a repo url when a dependent install happens -- it only
> updates
> > > the
> > > > >> meta-data.
> > > > >>
> > > > >> What I would like is a way (ideally within opam) to say "when this
> > > > >> package dependend-upon it should always be checked for update and
> > > upgrade".
> > > > >>
> > > > >> Am I correct in stating that currently there is no way to mark a
> > > package
> > > > >> as "update and upgrade this package whenever something that
> depends
> > > upon it
> > > > >> is installed"?
> > > > >>
> > > > >>
> > > > >> did you run `opam update -u <package>`? If a or dev or pinned
> package
> > > > >> changes it should normally trigger a recompilation of all the
> reverse
> > > > >> dependencies. how did you specify the packages in your repo?
> > > > >>
> > > > >> Thomas
> > > > >>
> > > > >>
> > > > >> Thanks.
> > > > >>
> > > > >> Trevor
> > > > >>
> > > > >> On Fri, May 8, 2015 at 2:24 PM, Daniel Bünzli <
> > > > >> daniel.buenzli at erratique.ch> wrote:
> > > > >>
> > > > >>> Le vendredi, 8 mai 2015 à 20:09, Ashish Agarwal a écrit :
> > > > >>> > Louis, thanks for your suggestions. I'm trying them out, but
> one
> > > quick
> > > > >>> question: how can you query with tags. I tried `opam list -e
> > > foobar`, and I
> > > > >>> seem to get the same output no matter what I write for foobar.
> > > > >>>
> > > > >>> This is not opam tags this is depexts tags (that correspond to
> > > > >>> platform). You can do for example:
> > > > >>>
> > > > >>>   opam search -s org:erratique
> > > > >>>
> > > > >>> But it may not be entirely precise since opam-search matches not
> > > only in
> > > > >>> tags. I think opam-list should be able to filter by tags (I
> actually
> > > > >>> thought this was possible).
> > > > >>>
> > > > >>> Best,
> > > > >>>
> > > > >>> Daniel
> > > > >>>
> > > > >>>
> > > > >>> _______________________________________________
> > > > >>> Platform mailing list
> > > > >>> Platform at lists.ocaml.org
> > > > >>> http://lists.ocaml.org/listinfo/platform
> > > > >>>
> > > > >>
> > > > >> _______________________________________________
> > > > >> Platform mailing list
> > > > >> Platform at lists.ocaml.org
> > > > >> http://lists.ocaml.org/listinfo/platform
> > > > >>
> > > > >>
> > > > >>
> > > > >
> > > > >
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ocaml.org/pipermail/platform/attachments/20150519/52eeb579/attachment-0001.html>