Skip to content

API documentation #125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
yarikoptic opened this issue Nov 30, 2017 · 14 comments
Closed

API documentation #125

yarikoptic opened this issue Nov 30, 2017 · 14 comments

Comments

@yarikoptic
Copy link

Would you be so kind to point me to some kind of API to the beast, so we could may be establish access to the "public" datasets available from https://web.gin.g-node.org via datalad, without coming up with some ad-hoc crawler ;)
Needed pieces would be to just

  • "protocol" to login and to "git clone"
  • discover the list of organization/datasets
  • fetch any available metadata associated with the datasets (authors, etc) if e.g. they are also "registered" (I see some at https://doid.gin.g-node.org/ ) or at least the doi (so we could query public services to provide that meta-data)

Thanks in advance!
@cgars
Copy link
Member

cgars commented Dec 1, 2017 •
edited
Loading

I guess you refer to the GIN Server, therefore the answer below is not client related:

Currently, we do not offer a specialized api routes for public or registered repositories (i might add them at some point in time). You can, however, use the search api as follows:
https://web.gin.g-node.org/api/v1/repos/search/?limit=1000
which will return all public repositories as json, including a json string with a ssh_url. This can be used for "ssh cloning".

Doified repositories contain a special file called either cloudberry.yml or datacite.yml which contains the bibliographic metadata used for registration. These can be queried as follows:
https://web.gin.g-node.org/INT/multielectrode_grasp/raw/master/cloudberry.yml
with the user and repository name changed accordingly. The files are in YAML format and should look like this. Not all repositories with such files are necessarily registered, however, and some (especially those from the doi user) will be forked repositories.

If you need more let me know and I can put it on either my to-implement or to-consider heap ;-).

@yarikoptic
Copy link
Author

Great! thank you @cgars , I will check all those out!

@yarikoptic
Copy link
Author

btw, that cloudberry.yml -- what is the "origin" of that specification? there are some cloudberry projects I googled up but not sure if they relate anyhow

@yarikoptic yarikoptic mentioned this issue Jul 21, 2021
Closed
@yarikoptic
Copy link
Author

@cgars @achilleas-k - is there an API end point for search? prior mentioned one is no longer there. Thanks in advance!

@yarikoptic yarikoptic reopened this Jul 21, 2021
@yarikoptic
Copy link
Author

FWIW there is https://github.com/gogs/docs-api which is probably what was used for the above /search endpoint which is no longer available.

@yarikoptic
Copy link
Author

woohoo, attn @jwodder, I found some more recent email from @achilleas-k with seems more up to date info:

https://gin.g-node.org/api/v1/repos/search?limit=50

See also https://github.com/gogs/docs-api/tree/master/Repositories#search-repositories
You can also do paging to get the list a few at a time.

@achilleas-k
Copy link
Member

Hi @yarikoptic, thanks for finding this. Didn't have a chance to catch up. The search you linked now will only match repository names, so I hope that's enough for now. The data search function doesn't have an API though.

@yarikoptic
Copy link
Author

oh... our greedy interest is to find repos which would have .datalad/config file in them. Name wouldn't be sufficient :-/

@yarikoptic
Copy link
Author

filed gogs/gogs#6594

@yarikoptic
Copy link
Author

Is API N/A temporarily or "for good"?:

❯ curl https://web.gin.g-node.org/api/v1/repos/search/?limit=10
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Unavailable</title>
</head><body>
<h1>Service Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
<p>Additionally, a 503 Service Unavailable
error was encountered while trying to use an ErrorDocument to handle the request.</p>
<hr>
<address>Apache/2.4.38 (Debian) Server at web.gin.g-node.org Port 443</address>
</body></html>

@yarikoptic yarikoptic mentioned this issue Apr 21, 2023
Closed
@twachtler
Copy link
Member

Hi @yarikoptic, please use https://gin.g-node.org/api/v1/repos/search/. "web.gin.g-node.org" urls are deprecated.

@yarikoptic
Copy link
Author

Thank you @twachtler for the rapid response! (attn @candleindark to ^^^).
I guess to list all repos we just need to iterate incrementing ?page= option until data becomes empty, right? or is there a better/recommended way?

@twachtler
Copy link
Member

I guess so. But please do it on the weekend ;)

@yarikoptic
Copy link
Author

deal! cron job then could run on weekends - no problem!

@jwodder jwodder mentioned this issue Feb 21, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@yarikoptic @cgars @achilleas-k @twachtler