Revamp --query

[felipecrs]

Closes #1148

@mxcl, I used Copilot to help me code this.

If you believe the code is not acceptable, feel free to just make changes yourself. There is no need to spend time reviewing this PR. If that's the case, I apologize for it. I'm really not familiar with the codebase and Rust.

I would like to highlight I wanted to reuse more of the existing input args parsing like +git +git@latest +git-scm.org^1, but I was unable to. That's probably the part which could use most refactoring.

Anyway, I changed a little bit how --query works. Let's go through the changes:

Unaltered behavior

Allows checking if pkgx can provide some program, and return project name if it can:

$ pkgx --query git; echo $?
git-scm.org
0

$ pkgx --query oops; echo $?
Error: "oops not found"
1

Altered behavior

All pkgspecs are now accepted, not only programs:

# old
$ pkgx --query openjdk.org; echo $?
openjdk.org not found
1

# new
$ pkgx --query openjdk.org
openjdk.org
0

If version is passed, it is validated to exist too:

$ pkgx --query git@1; echo $?
Error: "No versions matching @1 found for git-scm.org"
1

When no args are passed, returns all available projects (not programs):

# old
$ pkgx --query
...
zipinfo
zipmerge
zipnote
zippy
zipsplit
ziptool
zkCleanup
zkCli
zkEnv
zkServer
zkServer-initialize
zkSnapShotToolkit
zkSnapshotComparer
zkSnapshotRecursiveSummaryToolkit
zkTxnLogToolkit
zola
zoom2sam.pl
zoxide
zrok
zsh
zstd
zstdcat
zstdgrep
zstdless
zstdmt
zx

# new
$ pkgx --query
...
xcfile.dev
xerces.apache.org/xerces-c
xiph.org/flac
xiph.org/libshout
xkbcommon.org
xplr.dev
xpra.org
xtls.github.io
yadm.io
yarnpkg.com
yasm.tortall.net
youtube-dl.org
yt-dlp.org
yui.github.io/yuicompressor
zarf.dev
ziglang.org
zrok.io
zsh.sourceforge.io

The old behavior was inconsistent (when no args were provided programs were printed, but project was printed when args were provided). The old behavior can still be achieved like this:

# old
$ pkgx --query | wc -l
3843

# new
$ pkgx --query --json=v2 | jq -r '.[].programs[]' | sort | wc -l
3843

New behavior

--json=v2 now returns a JSON object with the following structure:

$ pkgx --query openjdk.org git --json=v2
[
  {
    "project": "openjdk.org",
    "programs": [
      "jar",
      "jarsigner",
      "java",
      "javac",
      "javadoc",
      "javap",
      "jcmd",
      "jconsole",
      "jdb",
      "jdeprscan",
      "jdeps",
      "jfr",
      "jhsdb",
      "jimage",
      "jinfo",
      "jlink",
      "jmap",
      "jmod",
      "jps",
      "jrunscript",
      "jshell",
      "jstack",
      "jstat",
      "jstatd",
      "keytool",
      "rmiregistry",
      "serialver"
    ]
  },
  {
    "project": "git-scm.org",
    "programs": [
      "git",
      "git-cvsserver",
      "git-receive-pack",
      "git-shell",
      "git-upload-archive",
      "git-upload-pack",
      "scalar"
    ]
  }
]

The output array follows the order of the arguments passed, and contains the resolved project name and their programs.

Flags can be passed at any position:

# old
$ pkgx --query java --silent; echo $?
openjdk.org
--silent not found
1

# new
$ pkgx --query java --silent; echo $?
0

Use case

I wrote my bash script to handle pkgx shims levering this new feature:

• https://github.com/felipecrs/dotfiles/blob/pkgxv2/home/dot_local/bin/executable_pkgs

And it works amazing:

❯ pkgs installed
maven.apache.org
opendev.org/git-review
openjdk.org@17

❯ pkgs installed git@2; echo $?
1

❯ pkgs installed-programs openjdk.org
...
jmod
jhsdb
jps
jdb
jmap
jstat

❯ pkgs uninstall java
uninstalling openjdk.org@17 (jdeps java jdeprscan jlink jconsole javac keytool serialver jimage jcmd jarsigner javadoc jstatd jstack rmiregistry jshell jrunscript javap jfr jinfo jar jmod jhsdb jps jdb jmap jstat)

❯ pkgs install git@2.31
Error: "No versions matching @2.31 found for git-scm.org"

❯ pkgs install git@2.44
installing git-scm.org@2.44 (git git-cvsserver git-receive-pack git-shell git-upload-archive git-upload-pack scalar)

❯ pkgs install git@2.45
uninstalling git-scm.org@2.44 (git-receive-pack git git-upload-archive git-cvsserver git-shell git-upload-pack scalar)
installing git-scm.org@2.45 (git git-cvsserver git-receive-pack git-shell git-upload-archive git-upload-pack scalar)

# nothing to uninstall
❯ pkgs uninstall git@2.44; echo $?
0

❯ pkgs uninstall git@2.45
uninstalling git-scm.org@2.45 (git-receive-pack git git-upload-archive git-cvsserver git-shell git-upload-pack scalar)

# reinstalls everything
❯ pkgs install
uninstalling git-scm.org@2.45 (git-receive-pack git git-upload-archive git-cvsserver git-shell git-upload-pack scalar)
installing git-scm.org@2.45 (git git-cvsserver git-receive-pack git-shell git-upload-archive git-upload-pack scalar)
uninstalling maven.apache.org (mvn mvnDebug mvnyjp)
installing maven.apache.org (mvn mvnDebug mvnyjp)
uninstalling opendev.org/git-review (git-review)
installing opendev.org/git-review (git-review)

# uninstalls everything
❯ pkgs uninstall
uninstalling git-scm.org@2.45 (git-receive-pack git git-upload-archive git-cvsserver git-shell git-upload-pack scalar)
uninstalling maven.apache.org (mvn mvnDebug mvnyjp)
uninstalling opendev.org/git-review (git-review)

Also, this pkgs script is able to recognize pkgx v1 stubs and gracefully refresh them, like when running pkgs install.

Honestly I don't want to keep it forever. The reasons why I wrote my own script are described here.

Someday maybe I will get around to porting it to Rust and maybe you can consider accepting it back. Maybe as some sort of new version of pkgm, or maybe as a whole new project that is focused on shims, so that pkgm can be left focused on "true" installations.

[felipecrs]

Thanks a lot for fixing the lint warnings, @jhheider!

---

@mxcl this is a gentle ping.

I'm using a self-built pkgx with this PR ever since I created it, and it's working like a charm.

It's probably not perfect, but if you are busy, maybe you can consider merging it and then we can enhance it later. I believe that's ok since it's only really touching the --query feature, which hopefully no one is using yet.

[jhheider]

@felipecrs let me see if i can't get this merged. the only one i don't like is changing the default behavior of the bare --query. i think you can easily make it work the way you changed it with a similar jq command in reverse.

[coveralls]

coverage: 89.873% (-0.5%) from 90.345%
when pulling c36fa49 on felipecrs:revamp-query
into 4e86ff3 on pkgxdev:main.

[felipecrs]

I understand. Breaking changes are never cool. But honestly, I highly doubt there's a person relying on it. mxcl added it upon my request, but IMO its previous behavior never made sense.

[felipecrs]

But if you really want, I can rework it tomorrow.

[jhheider]

my gut says people care more about the programs than what we've named the packages. but if working with the json becomes onerous, i'm not against a flag to give packages.

i'm digging into it now. no worries. thanks for getting the ball rolling.

[felipecrs]

my gut says people care more about the programs than what we've named the packages

That specifically doesn't change btw. Users can still query by program. With this PR they can also query by pkg name.

[jhheider]

i meant in terms of listing all. but i recognize it can be argued both ways.

[felipecrs]

@jhheider thank you so much! This is working like a charm.