[Esip-cor] Notifications

[Mcgibbney, Lewis J (398M)]

Hi John,


The LDN service is an entirely generic service, as I understand it, Anyone who wants to create a LDN inbox can do so; anyone who wants to publish to that inbox or listen on it can can do.

In the current implementation yes however, we should probably add basic authentication at some time in the future. Authentication is activated on the serverside however this is opaque for the sender or consumer of a LDN from the inbox.

If tomorrow I want a separate message queue (= inbox) for every ontology in COR—which makes perfect sense as a use case—I could take your software and make that happen.

Correct. If you wanted to create a new inbox however you would need to implement some trivial configuration changes.

(Or maybe I'd have to create a new deployment of it for each inbox?)

Yes, otherwise the default configuration would have you posting to the inbox service at http://cor.esipfed.org/ldn/inbox/ which is subsequently persisted as http://cor.esipfed.org/ont/ldn/inbox

The purpose of your current inbox is for testing, but potentially it is the first of many (many many).

Yes

It is easy to imagine a scaling issue.

It will take us a while to get the initiative scaling but yes if growth in the service increased then we would certainly need to act.

Hence my second point.

what happens if it is also populated with <insert large number of entities here>?
I don’t see this as an issue. People can still search for the resource they want to find via the GUI query bar.

Two constant refrains across MMI ORR, COR, and BioPortal, among other community portals I've run:
  1)  "I can't find my resource because there are so many of them."
  2)  "I thought about using your repository, but there's so much junk in there it doesn't look like a quality resource."

Some of these systems' improvements have directly targeted presenting useful resources to browsers and searchers, with limited success. But in the extreme case, if you don't know what string to search for ahead of time, and/or you get 10^x* responses for the string you do enter, it can be literally impossible to find what you need. And certainly the UI browser would be all but useless if the top 100 hits were notifications. These impacts could have direct consequences on adoption of COR by the community.

I do absolutely appreciate your concern John.
The good news is that the above does not represent a new use case and subsequent requirement for COR. It has been captured at https://esipfed.github.io/stc/UseCases/STCUseCasesAndRequirements.html#h-ontologybrowsing

To think about options, I think it's helpful to stipulate that an LDN RDF graph is a different kind of artifact in the COR, especially if it can be identified and managed as such by the system. For example, it isn't clear that showing the notification graphs to the front page UI is useful—there is no information in the title that's recognizable to a browser. Similarly, searching should be able to be restricted to LDN triples or non-LDN triples. With some software changes to COR, LDN could be a very interesting application in the COR context.

Yes I agree and I think if this can be logged into a Github issue then we can most likely address it in the GSoC project I proposed https://github.com/ESIPFed/gsoc/issues/12

But fundamentally, I think it's important we recognize that a notification system is just like a logging system: it needs to be built to handle a very large number of interactions. (Think of someone with a small-but-popular reprocessing tool that is logging status and error notifications every time the tool is run.

We considered this use case at ESIP when we met in January.

The log can become big quickly, especially if it is aggregating notifications from the whole world's use of that tool.) I know we aren't there now, but we should understand that implication as we discuss this rather cool technology.

Acknowledged.
This is good John. Thank you for scrutinizing the effort. It is appreciated.
Lewis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.esipfed.org/pipermail/esip-cor/attachments/20190206/7c7d96e2/attachment-0001.html>