Skip to content

[Feature Request]: Unique IDs for content-provider instances #632

@JAG-UK

Description

@JAG-UK

What is the problem you're trying to solve?

content-provider and download-server create relationships between SPs and data preparation sites (probably clients, but possibly someone working on the client's behalf).

In a vibrant storage ecosystem we should expect many-to-many relationships between these entities, and it would be great to be able to automate as much handling of these diverse sources and data with a robust way of identifying them - this for several reasons:

  • the one I actually hit recently: knowing where to fetch things from when there are multiple sources without doing super hacky things with passing direct URLs around
  • continuity of provenance even when IP addresses or similar move around
  • very large datasets where things might be prepared/hosted in shards
  • other reasons (I understand something like this was already in the vision "download-server mapping to multiple singularity instances" so I'm more adding weight to that than having a brand new idea)

On the surface this is very simple but we need to think carefully about how we do this since:

  • it has the potential to leak in unfortunate ways either into other application layers that maybe aren't 100% tied to Singularity
  • we need to think about whether size matters
  • we need to think about the issuing authority for these IDs and whether they're portable
  • we need to think about where there's value in cryptographically binding this ID to some other identifier that already exists in the protocol
  • we need to think about what level of granularity to use: per legal entity? per site? per invocation of content-provider? Hierarchical of all three?

IETF SPICE GLUE IDs might present a good syntax for this but still don't solve the questions above. Pure UUIDs (I like -4 and -7, but I can't present a slam-dunk case for either in this context) have some charm but introduce the need for a directory somewhere which would need maintenance.

Anyway, I've bodged around my immediate need for now so no huge urgency, but if others have touched this area before and are inspired to work on it I'm happy to help with design and testing.

Describe the workaround you currently have

I don't have any workaround

Describe the feature you'd like

I would like singularity to ...

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions