Load and update strategy¶

As you incorporate packages and their dependencies into your application, different packages sometimes specify different versions of the same package as dependencies. As a package manager, Tatin has policies for resolving these conflicts.

You can get started without grasping these policies, but they have important consequences you will eventually need to understand.

Minimum version selection¶

Suppose you need packages, Foo and Goo. Both rely on package Zoo; but while Foo requests Zoo 1.1.1, Goo requires version 1.2.0.

         Zoo-1.0.0
         Zoo-1.1.0
Foo ---> Zoo-1.1.1
Goo ---> Zoo-1.2.0
         Zoo-1.3.0
         Zoo-2.0.0

The best available version 1 of Zoo is 1.3.0. There is also a later major version: 2.0.0.

Version 2.0.0 is not an option, because it is considered a completely different package and is therefore ignored.

From semantic versioning we know Goo relies on features of Zoo new in version 1.2.0, and would break on version 1.1.1, which doesn’t have them.

Strategy options¶

Several strategies are possible:

1. Load both versions of Zoo: let Foo use 1.1.1 and Goo 1.2.0.
2. Load Zoo version 1.2.0 and let Foo and Goo both use it.
3. Check the server, load the latest version available (1.3.0) and use that.

Each of these options is used by different package managers in the wild. There is no consensus on what is best.

Strategy (3) appeals to many programmers: you are always up-to-date, and you always get the latest fixes when you build your application. Great!

But on reflection it looks much less appealing:

1. Build your application, including updated dependencies
2. Run your test cases; they all pass
3. Rebuild your application without changing anything
4. Rerun your test cases; but this time they fail

This situation arises if, between the first and second build, the author of a dependency releases a new version with bugs in it. The second build fetches it, producing a different outcome – even though you haven’t made any changes.

Tatin’s strategy¶

As appealing as an automated update mechanism might seem, you’d want your builds to be 100% reproducible, right?

So, when asked to load installed packages, Tatin will do exactly that:

• load the packages defined as required by the configuration files of the main packages Foo and Goo
• except when a package is requested more than once, and with different minor and/or patch numbers, Tatin uses the latest installed version (which might or might not be the latest one available)

In our example

• Zoo is requested and installed twice
• Tatin loads 1.2.0 just once (because it is requested by Goo)
• 1.3.0 is ignored because it is not requested and so not installed

This means Foo will also use version 1.2.0 of Zoo.

This strategy is called Minimal Version Selection (MVS). It guarantees that when you rebuild, you get the same result, because it grabs the latest installed version.

Forcing updates¶

Suppose a package of yours depends on someone else’s package Foo 3.0.0, which in turn depends on Goo 1.1.0 – and while using Foo you discover Goo 1.1.0 has a bug.

You check and find that the author of Goo has fixed the bug and released the fix as 1.1.1. So you write to the author of Foo, saying Goo 1.1.1 is available and that they should release Foo 3.0.1 with an updated dependency, but you get no reply, and you have little time left to your deadline.

Solution: Create a package Dummy that does nothing, but declares a dependency on Goo 1.1.1, and add Dummy as a dependency to your package. Voilà! Thanks to MVS, everything works.

Comparing versions¶

As long as the patch is a number, precedence is easy to determine.

Precedence is obvious for

group-name-1.0.0
group-name-1.1.0

less so for

group-name-1.0.0-alpha
group-name-1.1.0-beta

and cannot be determined with

group-name-1.0.0-TryFeature1
group-name-1.1.0-FixFor234

Packages, once published, cannot be altered. So if the patch contains a text description, Tatin uses publication date to determine the correct order.

Loading dependencies¶

Imagine these packages all hosted by a Tatin server with alias MyTatin.

mygroup-Foo-1.0.0 ---> Zoo-1.1.1
mygroup-Goo-2.1.0 ---> Zoo-1.2.0
                       Zoo-1.3.0

While Foo relies on Zoo 1.1.1, Goo needs Zoo 1.2.0. Version 1.3.0 of Zoo is also available. What does Tatin do about this?

It depends on what you do. Let’s first load Foo and Goo into the workspace, the usual way to inspect a package:

      ]TATIN.LoadPackages [MyTatin]/mygroup-Foo-1.0.0 #.MyPkgs
#.MyPkgs.Foo
      ]TATIN.LoadPackages [MyTatin]/mygroup-Goo-2.1.0 #.MyPkgs
#.MyPkgs.Goo
      #.MyPkgs.⎕NL ⍳16
Foo
Goo

Getting the latest version¶

Suppose version 1.0.0 of Foo is the latest available with major number 1, then this would have been sufficient:

]TATIN.LoadPackages [MyTatin]/mygroup-Foo-1 #.MyPkgs

Note that neither the minor nor the patch number has been specified.

Suppose version 2.1.0 of Goo is the very latest version available, and that’s the version you want to use, then this would have been sufficient:

]TATIN.LoadPackages [MyTatin]/mygroup-Goo #.MyPkgs

In this case not even the major number has been specified.

You wanted Foo and Goo loaded into #.MyPkgs, and that's exactly what Tatin did. But where are the dependencies?

Looking under the covers:

      #._tatin.⎕NL 9
mygroup_Foo_1_0_0
mygroup_Goo_2_1_0
mygroup_Zoo_1_1_1
mygroup_Zoo_1_2_0

Both versions of Zoo have been loaded. That’s because the two Load operations are independent, so minimal version selection cannot be applied.

Installing packages¶

To incorporate packages into an application, they must be installed. In the following example, we install two packages, Foo and Goo. Both require Zoo, so Zoo gets installed as a side effect.

      ⎕NEXISTS '/myPkgs'
0
      ]TATIN.InstallPackages [MyTatin]/mygroup-Foo-1.0.0 /myPkgs/
/myPkgs/mygroup-Foo-1.0.0
      ]TATIN.InstallPackages [MyTatin]/mygroup-Goo-2.1.0 /myPkgs/
/myPkgs/mygroup-Goo-2.1.0
      ⍪⊃⎕NINFO ⍠ 1⊢'/myPkgs/*'
/myPkgs/apl-buildlist.json
/myPkgs/apl-dependencies.txt
/myPkgs/mygroup-Foo-1.0.0
/myPkgs/mygroup-Goo-2.1.0
/myPkgs/mygroup-Zoo-1.1.1
/myPkgs/mygroup-Zoo-1.2.0

Compare the dependency file and the build list:

      ⊃⎕NGET '/myPkgs/apl-dependencies.txt'    ⍝ dependency file
mygroup-Foo-1.0.0
mygroup-Goo-2.1.0

      json←⊃⎕NGET'/myPkgs/apl-buildlist.json'  ⍝ build list
      #.q←⎕JSON ⍠ ('Dialect' 'JSON5') ⊢ json
      ⍉↑#.q.(principal packageID)
1  mygroup-Foo-1.0.0
0  mygroup-Zoo-1.1.1
1  mygroup-Goo-2.1.0
0  mygroup-Zoo-1.2.0

Above, the first column flags the principal packages.

• The dependency file lists the principal packages installed.
• The build list comprises not only the two principal packages but also the dependencies.

Loading installed packages¶

Loading packages puts them in your workspace; installing packages puts them in your file system.

Loading installed packages lets Tatin use MVS to optimise what it loads.

Load installed packages with command ]LoadDependencies. Its argument is a folder with files apl-dependencies.txt and apl-buildlist.json, created when you installed your first package there.

You can also specify a second argument: where to create the references pointing to the principal packages. It defaults to # if that is your current namespace. If you are, say, in #.foo, then Tatin asks whether to load into # or #.foo.

]TATIN.LoadDependencies /myPkgs/ #.Temp

If the command finds the dependency file is newer than the build list, it checks for inconsistencies: if none, it reconstructs the build list from scratch.

Then it prunes the build list and loads the remaining packages.

In our example the pruning eliminates Zoo 1.1.1: rather than loading Zoo twice, Tatin loads only the latest installed version. Both Foo and Goo will use Zoo version 1.2.0.

Upgrading to a new major version¶

One way to upgrade to a new major version is to delete all packages from an installation folder, change the dependency file to refer to the new major version number and then open the project with Cider.

Cider will detect the need for re-installing and act accordingly.

Remove a principal package¶

Suppose you no longer need Goo.

To remove it, use either command ]UnInstallPackage or API function UnInstallPackage. This will remove not only the principal package, but any of its dependencies that are neither principal packages nor dependencies of other packages.

Implicit downgrading¶

Removing a principal package might imply a downgrade.

Consider, say, these packages (dependencies indented):

group-Foo-1.0.0
  group-Zoo-1.1.0
group-Boo-1.0.0
  group-Zoo-1.0.0

Minimal version selection (MVS) means both Foo and Boo use Zoo 1.1.0 when loaded. What happens if you remove Boo?

One could argue either:

1. Zoo-1.1.0 should survive this and still be used
2. a package not required anywhere shouldn’t be used

A full implementation of MVS requires (1). For now, Tatin uses (2) as a more pragmatic approach.

It is not obvious what is best here. Tatin’s strategy might change after feedback.

Add a package manually¶

To add a package that has no dependencies you could just add it to the file apl-dependencies.txt and copy the package over.

That would not cause a problem. When the LoadDependencies command is next issued, Tatin sees the file apl-dependencies.txt changed after apl-buildlist.json was last modified, performs some health checks, and if all is well, recreates the build list. (This is insurance against people manually introducing errors into apl-dependencies.txt)

Downgrade¶

Sometimes you might need to downgrade, for example from a version you find buggy, to an older version known to work.

Tatin does not offer help here; you need to do this yourself.