[git-buildpackage] `gbp import-orig --pristine-tar` with component tarballs

[Nish Aravamudan]

Hello,

I *think* I may have found a bug in `gbp import-orig --pristine-tar` with
multiple component tarballs, but I'm not 100% :)

I'm specifically looking (as a test example) src:gosa, e.g.,
2.7.4+reloaded2-13. The behavior I'm seeing is that while the machine
that does the pristine-tar import is able to re-generate the tarballs,
if I `git push` all my branches and then `git clone` that same
repository down, the new repository is not able to (the tree IDs in the
various .id files are not present in the clone repository). I think this
only happens with multiple component tarballs, because `gbp import-orig`
does:

scripts/import_orig.py - main
   ...
   commit = repo.commit_dir(source.unpacked,
                            ...
                            branch=import_branch,
                            ...
   ...

where source.unpacked is the fully unpacked main + component tarballs

   source, tmpdir = unpack_tarballs(sourcepackage, source, version, component_tarballs, options)

This commit is later tagged by the --upstream-tag value, so even if we
were to throw away an upstream branch (e.g., not push it), in theory we
have the corresponding treeish via this commit.

It's relevant to note that the tree of this commit does not correspond
to any actual tarball, but the combination of all tarballs (as needed to
build the package).

However,

deb/git.py - DebianGitRepository::create_pristinetar_commits

does:

        components = [c for (c, t) in component_tarballs]
        main_tree = self.tree_drop_dirs(upstream_tree, components)

        for component, name in component_tarballs:
            subtree = self.tree_get_dir(upstream_tree, component)
            if not subtree:
                raise GitRepositoryError("No tree for '%s' found in '%s' to create pristine tar commit from" % (component, upstream_tree))
            gbp.log.debug("Creating pristine tar commit '%s' from '%s'" % (component, subtree))
            self.pristine_tar.commit(name, subtree)
        self.pristine_tar.commit(tarball, main_tree)

These pristine-tar commits, which get put on the pristine-tar branch
will, potentially [0], correspond to trees that are not present on the
upstream branch at all, because they are for each tarball's contents (as
represented by the set of main_tree and all subtrees).

I am not sure how this would ever work -- I know the trees as contained
by the .id files in the pristine-tar branches must be resolveable by
git. But I also assume the intentions is that one can do this
--component import and push like usual and be able to reproduce all the
tarballs. Perhaps I am missing something obvious!

I completely understand that this is an experimental and new feature,
but I would love to leverage it in our Ubuntu source package importer
(as I don't want to deal with all the corner cases gbp already does in
our code for handling pristine-tar :) -- so if I am on the right track
and patches would be welcome, then I am happy to work on something! [1] 

Thanks,
-Nish

[0] I say potentially, because I think if there are no component
tarballs, the main_tree value will correspond exactly to the tree of the
commit from main earlier.

[1] I see
https://github.com/agx/git-buildpackage/commit/956982c52500ff52bdab6bb811b86ad6fedbaa19
was recently committed, and I'm reading it now to see if it's covering
exactly this case. I think it does not, because that seems to mostly
just be a wrapper around create_pristinetar_commits, which will end up
storing trees in the .id files for pristine-tar that are not reachable
in the repository.

-- 
Nishanth Aravamudan
Ubuntu Server
Canonical Ltd