CAMEL 13894: Pulling Contributor data from JSON APIs

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

CAMEL 13894: Pulling Contributor data from JSON APIs

Cynthia Omovoiye
Hi,
I am Cynthia Omovoiye, an applicant for Outreachy internship and I picked
interest in the Apache Camel site project with CAMEL 13894 as my main
focus.

Although I am quite new to this topic, I am ready to learn and contribute
to this project. Please, I would like to know what is expected of me in the
completion of this project? I understand that I have to update the database
of Contributors by pulling in their data through an API. I, however, don't
know where I am required to pull it from and where I am supposed to upload
it to.

Please, assist me with this area. Thank you for your assistance.
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Zoran Regvart-2
Hi Cynthia,
on the Camel website we have a page where we list all the team members:

https://camel.apache.org/community/team/

I think we can get the data from that page from the
projects.apache.org, for example this page:

https://projects.apache.org/committee.html?camel

Lists all the PMC members (PMC Roster) and the Comitters, and I think
the data is fetched from JSON at:

https://projects.apache.org/json/foundation/committees.json

There's a lot of data in JSON files here, that you can also take a look at:

https://projects.apache.org/json/

We also have Contributors, folk much like you, that aren't associated
with Apache Software Foundation but help out from time to time and
submit changes via GitHub. GitHub also offers this data as JSON, here
is the documentation on how to fetch the list of all contributors:

https://developer.github.com/v3/repos/collaborators/#list-collaborators

zoran

On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <[hidden email]> wrote:

>
> Hi,
> I am Cynthia Omovoiye, an applicant for Outreachy internship and I picked
> interest in the Apache Camel site project with CAMEL 13894 as my main
> focus.
>
> Although I am quite new to this topic, I am ready to learn and contribute
> to this project. Please, I would like to know what is expected of me in the
> completion of this project? I understand that I have to update the database
> of Contributors by pulling in their data through an API. I, however, don't
> know where I am required to pull it from and where I am supposed to upload
> it to.
>
> Please, assist me with this area. Thank you for your assistance.



--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Cynthia Omovoiye
Hi Zoran,

I have gone through the JSON files you provided and I would like to know
where I can find the organization names as displayed on the table on the
team page.

Thank you,
Cynthia.

On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]> wrote:

> Hi Cynthia,
> on the Camel website we have a page where we list all the team members:
>
> https://camel.apache.org/community/team/
>
> I think we can get the data from that page from the
> projects.apache.org, for example this page:
>
> https://projects.apache.org/committee.html?camel
>
> Lists all the PMC members (PMC Roster) and the Comitters, and I think
> the data is fetched from JSON at:
>
> https://projects.apache.org/json/foundation/committees.json
>
> There's a lot of data in JSON files here, that you can also take a look at:
>
> https://projects.apache.org/json/
>
> We also have Contributors, folk much like you, that aren't associated
> with Apache Software Foundation but help out from time to time and
> submit changes via GitHub. GitHub also offers this data as JSON, here
> is the documentation on how to fetch the list of all contributors:
>
> https://developer.github.com/v3/repos/collaborators/#list-collaborators
>
> zoran
>
> On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <[hidden email]>
> wrote:
> >
> > Hi,
> > I am Cynthia Omovoiye, an applicant for Outreachy internship and I picked
> > interest in the Apache Camel site project with CAMEL 13894 as my main
> > focus.
> >
> > Although I am quite new to this topic, I am ready to learn and contribute
> > to this project. Please, I would like to know what is expected of me in
> the
> > completion of this project? I understand that I have to update the
> database
> > of Contributors by pulling in their data through an API. I, however,
> don't
> > know where I am required to pull it from and where I am supposed to
> upload
> > it to.
> >
> > Please, assist me with this area. Thank you for your assistance.
>
>
>
> --
> Zoran Regvart
>
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Omar Al-Safi
HI Cynthia,

The organization names are being entered manually in the GitHub repository
<https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md>,
perhaps you can scrap that page manually to extract this information?

Regards,
Omar

On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]>
wrote:

> Hi Zoran,
>
> I have gone through the JSON files you provided and I would like to know
> where I can find the organization names as displayed on the table on the
> team page.
>
> Thank you,
> Cynthia.
>
> On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]> wrote:
>
> > Hi Cynthia,
> > on the Camel website we have a page where we list all the team members:
> >
> > https://camel.apache.org/community/team/
> >
> > I think we can get the data from that page from the
> > projects.apache.org, for example this page:
> >
> > https://projects.apache.org/committee.html?camel
> >
> > Lists all the PMC members (PMC Roster) and the Comitters, and I think
> > the data is fetched from JSON at:
> >
> > https://projects.apache.org/json/foundation/committees.json
> >
> > There's a lot of data in JSON files here, that you can also take a look
> at:
> >
> > https://projects.apache.org/json/
> >
> > We also have Contributors, folk much like you, that aren't associated
> > with Apache Software Foundation but help out from time to time and
> > submit changes via GitHub. GitHub also offers this data as JSON, here
> > is the documentation on how to fetch the list of all contributors:
> >
> > https://developer.github.com/v3/repos/collaborators/#list-collaborators
> >
> > zoran
> >
> > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <[hidden email]>
> > wrote:
> > >
> > > Hi,
> > > I am Cynthia Omovoiye, an applicant for Outreachy internship and I
> picked
> > > interest in the Apache Camel site project with CAMEL 13894 as my main
> > > focus.
> > >
> > > Although I am quite new to this topic, I am ready to learn and
> contribute
> > > to this project. Please, I would like to know what is expected of me in
> > the
> > > completion of this project? I understand that I have to update the
> > database
> > > of Contributors by pulling in their data through an API. I, however,
> > don't
> > > know where I am required to pull it from and where I am supposed to
> > upload
> > > it to.
> > >
> > > Please, assist me with this area. Thank you for your assistance.
> >
> >
> >
> > --
> > Zoran Regvart
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Cynthia Omovoiye
Hi Zoran,
I have been able to pull all required data from the JSON files into the team page for Committers as shown in the screenshot below(committers.png), I have also been able to set the GitHub API that extracts the collaborators data from the repository and rendered it to display on the team page. However, it will require a GitHub access token from the Apache GitHub account for the data to be displayed but I was able to generate one on my forked repository of the Apache camel website for testing which returned the data displayed in the screenshot below(contributors.png). Thus, if an access token  generated by an admin of the Apache camel website repository is inputted in the field I have made available for it in(" /layouts/shortcodes/contributors.html " {{$token :=" "}}), we would be able get the data of the collaborators  displayed on the web page.

On Fri, Mar 20, 2020 at 9:44 AM Omar Al-Safi <[hidden email]> wrote:
HI Cynthia,

The organization names are being entered manually in the GitHub repository
<https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md>,
perhaps you can scrap that page manually to extract this information?

Regards,
Omar

On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]>
wrote:

> Hi Zoran,
>
> I have gone through the JSON files you provided and I would like to know
> where I can find the organization names as displayed on the table on the
> team page.
>
> Thank you,
> Cynthia.
>
> On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]> wrote:
>
> > Hi Cynthia,
> > on the Camel website we have a page where we list all the team members:
> >
> > https://camel.apache.org/community/team/
> >
> > I think we can get the data from that page from the
> > projects.apache.org, for example this page:
> >
> > https://projects.apache.org/committee.html?camel
> >
> > Lists all the PMC members (PMC Roster) and the Comitters, and I think
> > the data is fetched from JSON at:
> >
> > https://projects.apache.org/json/foundation/committees.json
> >
> > There's a lot of data in JSON files here, that you can also take a look
> at:
> >
> > https://projects.apache.org/json/
> >
> > We also have Contributors, folk much like you, that aren't associated
> > with Apache Software Foundation but help out from time to time and
> > submit changes via GitHub. GitHub also offers this data as JSON, here
> > is the documentation on how to fetch the list of all contributors:
> >
> > https://developer.github.com/v3/repos/collaborators/#list-collaborators
> >
> > zoran
> >
> > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <[hidden email]>
> > wrote:
> > >
> > > Hi,
> > > I am Cynthia Omovoiye, an applicant for Outreachy internship and I
> picked
> > > interest in the Apache Camel site project with CAMEL 13894 as my main
> > > focus.
> > >
> > > Although I am quite new to this topic, I am ready to learn and
> contribute
> > > to this project. Please, I would like to know what is expected of me in
> > the
> > > completion of this project? I understand that I have to update the
> > database
> > > of Contributors by pulling in their data through an API. I, however,
> > don't
> > > know where I am required to pull it from and where I am supposed to
> > upload
> > > it to.
> > >
> > > Please, assist me with this area. Thank you for your assistance.
> >
> >
> >
> > --
> > Zoran Regvart
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Zoran Regvart-2
Hi Cynthia,
I've found that the contributors API[1] perhaps the contributors API
doesn't require authentication. Though we might end up hitting the
rate limits and then requiring the token. We can configure that on the
build server, for example we can provide the token via an environment
variable.

What I think will be difficult is that with the responses from GitHub,
there is usually a limit on the number of results returned. So most
likely there will be a need for paginating the results. Not sure how
difficult that would be in Hugo.

I noticed that there's a JSON data file in your changes, I would try
to avoid that if possible and fetch the data dynamically.

It would be easier to have this conversation on a GitHub pull request,
so you might want to consider creating one for this.

zoran

[1] https://developer.github.com/v3/repos/#list-contributors

On Wed, Mar 25, 2020 at 8:35 PM Cynthia Omovoiye <[hidden email]> wrote:

>
> Hi Zoran,
> I have been able to pull all required data from the JSON files into the team page for Committers as shown in the screenshot below(committers.png), I have also been able to set the GitHub API that extracts the collaborators data from the repository and rendered it to display on the team page. However, it will require a GitHub access token from the Apache GitHub account for the data to be displayed but I was able to generate one on my forked repository of the Apache camel website for testing which returned the data displayed in the screenshot below(contributors.png). Thus, if an access token  generated by an admin of the Apache camel website repository is inputted in the field I have made available for it in(" /layouts/shortcodes/contributors.html " {{$token :=" "}}), we would be able get the data of the collaborators  displayed on the web page.
> https://github.com/CynthiaOmovoiye/camel-website/tree/pullingDataFromJsonApis
>
> On Fri, Mar 20, 2020 at 9:44 AM Omar Al-Safi <[hidden email]> wrote:
>>
>> HI Cynthia,
>>
>> The organization names are being entered manually in the GitHub repository
>> <https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md>,
>> perhaps you can scrap that page manually to extract this information?
>>
>> Regards,
>> Omar
>>
>> On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]>
>> wrote:
>>
>> > Hi Zoran,
>> >
>> > I have gone through the JSON files you provided and I would like to know
>> > where I can find the organization names as displayed on the table on the
>> > team page.
>> >
>> > Thank you,
>> > Cynthia.
>> >
>> > On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]> wrote:
>> >
>> > > Hi Cynthia,
>> > > on the Camel website we have a page where we list all the team members:
>> > >
>> > > https://camel.apache.org/community/team/
>> > >
>> > > I think we can get the data from that page from the
>> > > projects.apache.org, for example this page:
>> > >
>> > > https://projects.apache.org/committee.html?camel
>> > >
>> > > Lists all the PMC members (PMC Roster) and the Comitters, and I think
>> > > the data is fetched from JSON at:
>> > >
>> > > https://projects.apache.org/json/foundation/committees.json
>> > >
>> > > There's a lot of data in JSON files here, that you can also take a look
>> > at:
>> > >
>> > > https://projects.apache.org/json/
>> > >
>> > > We also have Contributors, folk much like you, that aren't associated
>> > > with Apache Software Foundation but help out from time to time and
>> > > submit changes via GitHub. GitHub also offers this data as JSON, here
>> > > is the documentation on how to fetch the list of all contributors:
>> > >
>> > > https://developer.github.com/v3/repos/collaborators/#list-collaborators
>> > >
>> > > zoran
>> > >
>> > > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <[hidden email]>
>> > > wrote:
>> > > >
>> > > > Hi,
>> > > > I am Cynthia Omovoiye, an applicant for Outreachy internship and I
>> > picked
>> > > > interest in the Apache Camel site project with CAMEL 13894 as my main
>> > > > focus.
>> > > >
>> > > > Although I am quite new to this topic, I am ready to learn and
>> > contribute
>> > > > to this project. Please, I would like to know what is expected of me in
>> > > the
>> > > > completion of this project? I understand that I have to update the
>> > > database
>> > > > of Contributors by pulling in their data through an API. I, however,
>> > > don't
>> > > > know where I am required to pull it from and where I am supposed to
>> > > upload
>> > > > it to.
>> > > >
>> > > > Please, assist me with this area. Thank you for your assistance.
>> > >
>> > >
>> > >
>> > > --
>> > > Zoran Regvart
>> > >
>> >



--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Cynthia Omovoiye
Hi Zoran,
Thank you for your feedback. I will start working on it immediately.

I have already created a pull request on GitHub.
https://github.com/apache/camel-website/pull/272

With regards,
Cynthia.

On Thu, Mar 26, 2020, 10:09 AM Zoran Regvart <[hidden email]> wrote:

> Hi Cynthia,
> I've found that the contributors API[1] perhaps the contributors API
> doesn't require authentication. Though we might end up hitting the
> rate limits and then requiring the token. We can configure that on the
> build server, for example we can provide the token via an environment
> variable.
>
> What I think will be difficult is that with the responses from GitHub,
> there is usually a limit on the number of results returned. So most
> likely there will be a need for paginating the results. Not sure how
> difficult that would be in Hugo.
>
> I noticed that there's a JSON data file in your changes, I would try
> to avoid that if possible and fetch the data dynamically.
>
> It would be easier to have this conversation on a GitHub pull request,
> so you might want to consider creating one for this.
>
> zoran
>
> [1] https://developer.github.com/v3/repos/#list-contributors
>
> On Wed, Mar 25, 2020 at 8:35 PM Cynthia Omovoiye <[hidden email]>
> wrote:
> >
> > Hi Zoran,
> > I have been able to pull all required data from the JSON files into the
> team page for Committers as shown in the screenshot below(committers.png),
> I have also been able to set the GitHub API that extracts the collaborators
> data from the repository and rendered it to display on the team page.
> However, it will require a GitHub access token from the Apache GitHub
> account for the data to be displayed but I was able to generate one on my
> forked repository of the Apache camel website for testing which returned
> the data displayed in the screenshot below(contributors.png). Thus, if an
> access token  generated by an admin of the Apache camel website repository
> is inputted in the field I have made available for it in("
> /layouts/shortcodes/contributors.html " {{$token :=" "}}), we would be able
> get the data of the collaborators  displayed on the web page.
> >
> https://github.com/CynthiaOmovoiye/camel-website/tree/pullingDataFromJsonApis
> >
> > On Fri, Mar 20, 2020 at 9:44 AM Omar Al-Safi <[hidden email]> wrote:
> >>
> >> HI Cynthia,
> >>
> >> The organization names are being entered manually in the GitHub
> repository
> >> <
> https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md
> >,
> >> perhaps you can scrap that page manually to extract this information?
> >>
> >> Regards,
> >> Omar
> >>
> >> On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]>
> >> wrote:
> >>
> >> > Hi Zoran,
> >> >
> >> > I have gone through the JSON files you provided and I would like to
> know
> >> > where I can find the organization names as displayed on the table on
> the
> >> > team page.
> >> >
> >> > Thank you,
> >> > Cynthia.
> >> >
> >> > On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]>
> wrote:
> >> >
> >> > > Hi Cynthia,
> >> > > on the Camel website we have a page where we list all the team
> members:
> >> > >
> >> > > https://camel.apache.org/community/team/
> >> > >
> >> > > I think we can get the data from that page from the
> >> > > projects.apache.org, for example this page:
> >> > >
> >> > > https://projects.apache.org/committee.html?camel
> >> > >
> >> > > Lists all the PMC members (PMC Roster) and the Comitters, and I
> think
> >> > > the data is fetched from JSON at:
> >> > >
> >> > > https://projects.apache.org/json/foundation/committees.json
> >> > >
> >> > > There's a lot of data in JSON files here, that you can also take a
> look
> >> > at:
> >> > >
> >> > > https://projects.apache.org/json/
> >> > >
> >> > > We also have Contributors, folk much like you, that aren't
> associated
> >> > > with Apache Software Foundation but help out from time to time and
> >> > > submit changes via GitHub. GitHub also offers this data as JSON,
> here
> >> > > is the documentation on how to fetch the list of all contributors:
> >> > >
> >> > >
> https://developer.github.com/v3/repos/collaborators/#list-collaborators
> >> > >
> >> > > zoran
> >> > >
> >> > > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <[hidden email]
> >
> >> > > wrote:
> >> > > >
> >> > > > Hi,
> >> > > > I am Cynthia Omovoiye, an applicant for Outreachy internship and I
> >> > picked
> >> > > > interest in the Apache Camel site project with CAMEL 13894 as my
> main
> >> > > > focus.
> >> > > >
> >> > > > Although I am quite new to this topic, I am ready to learn and
> >> > contribute
> >> > > > to this project. Please, I would like to know what is expected of
> me in
> >> > > the
> >> > > > completion of this project? I understand that I have to update the
> >> > > database
> >> > > > of Contributors by pulling in their data through an API. I,
> however,
> >> > > don't
> >> > > > know where I am required to pull it from and where I am supposed
> to
> >> > > upload
> >> > > > it to.
> >> > > >
> >> > > > Please, assist me with this area. Thank you for your assistance.
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Zoran Regvart
> >> > >
> >> >
>
>
>
> --
> Zoran Regvart
>
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Cynthia Omovoiye
In reply to this post by Zoran Regvart-2
Hi Zoran,
Thank you for your feedback, I have been able to effect your corrections and I have been able to pull the Contributors data and have it display on the page but I am concerned that the information[contributors3.jpg] retrieved from this data do not correspond with the information[contributors2.jpg] already listed in the Contributors section on the team.md page. Is there somewhere else to pull this data from or can I go on with what I have? Thank you.




With kind regards,
Cynthia.


On Thu, Mar 26, 2020, 10:09 AM Zoran Regvart <[hidden email]> wrote:
Hi Cynthia,
I've found that the contributors API[1] perhaps the contributors API
doesn't require authentication. Though we might end up hitting the
rate limits and then requiring the token. We can configure that on the
build server, for example we can provide the token via an environment
variable.

What I think will be difficult is that with the responses from GitHub,
there is usually a limit on the number of results returned. So most
likely there will be a need for paginating the results. Not sure how
difficult that would be in Hugo.

I noticed that there's a JSON data file in your changes, I would try
to avoid that if possible and fetch the data dynamically.

It would be easier to have this conversation on a GitHub pull request,
so you might want to consider creating one for this.

zoran

[1] https://developer.github.com/v3/repos/#list-contributors

On Wed, Mar 25, 2020 at 8:35 PM Cynthia Omovoiye <[hidden email]> wrote:
>
> Hi Zoran,
> I have been able to pull all required data from the JSON files into the team page for Committers as shown in the screenshot below(committers.png), I have also been able to set the GitHub API that extracts the collaborators data from the repository and rendered it to display on the team page. However, it will require a GitHub access token from the Apache GitHub account for the data to be displayed but I was able to generate one on my forked repository of the Apache camel website for testing which returned the data displayed in the screenshot below(contributors.png). Thus, if an access token  generated by an admin of the Apache camel website repository is inputted in the field I have made available for it in(" /layouts/shortcodes/contributors.html " {{$token :=" "}}), we would be able get the data of the collaborators  displayed on the web page.
> https://github.com/CynthiaOmovoiye/camel-website/tree/pullingDataFromJsonApis
>
> On Fri, Mar 20, 2020 at 9:44 AM Omar Al-Safi <[hidden email]> wrote:
>>
>> HI Cynthia,
>>
>> The organization names are being entered manually in the GitHub repository
>> <https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md>,
>> perhaps you can scrap that page manually to extract this information?
>>
>> Regards,
>> Omar
>>
>> On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]>
>> wrote:
>>
>> > Hi Zoran,
>> >
>> > I have gone through the JSON files you provided and I would like to know
>> > where I can find the organization names as displayed on the table on the
>> > team page.
>> >
>> > Thank you,
>> > Cynthia.
>> >
>> > On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]> wrote:
>> >
>> > > Hi Cynthia,
>> > > on the Camel website we have a page where we list all the team members:
>> > >
>> > > https://camel.apache.org/community/team/
>> > >
>> > > I think we can get the data from that page from the
>> > > projects.apache.org, for example this page:
>> > >
>> > > https://projects.apache.org/committee.html?camel
>> > >
>> > > Lists all the PMC members (PMC Roster) and the Comitters, and I think
>> > > the data is fetched from JSON at:
>> > >
>> > > https://projects.apache.org/json/foundation/committees.json
>> > >
>> > > There's a lot of data in JSON files here, that you can also take a look
>> > at:
>> > >
>> > > https://projects.apache.org/json/
>> > >
>> > > We also have Contributors, folk much like you, that aren't associated
>> > > with Apache Software Foundation but help out from time to time and
>> > > submit changes via GitHub. GitHub also offers this data as JSON, here
>> > > is the documentation on how to fetch the list of all contributors:
>> > >
>> > > https://developer.github.com/v3/repos/collaborators/#list-collaborators
>> > >
>> > > zoran
>> > >
>> > > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <[hidden email]>
>> > > wrote:
>> > > >
>> > > > Hi,
>> > > > I am Cynthia Omovoiye, an applicant for Outreachy internship and I
>> > picked
>> > > > interest in the Apache Camel site project with CAMEL 13894 as my main
>> > > > focus.
>> > > >
>> > > > Although I am quite new to this topic, I am ready to learn and
>> > contribute
>> > > > to this project. Please, I would like to know what is expected of me in
>> > > the
>> > > > completion of this project? I understand that I have to update the
>> > > database
>> > > > of Contributors by pulling in their data through an API. I, however,
>> > > don't
>> > > > know where I am required to pull it from and where I am supposed to
>> > > upload
>> > > > it to.
>> > > >
>> > > > Please, assist me with this area. Thank you for your assistance.
>> > >
>> > >
>> > >
>> > > --
>> > > Zoran Regvart
>> > >
>> >



--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Cynthia Omovoiye
Hi Zoran,

Please disregard the previous email, I have been able to sort the issue out.
Sorry for any inconvenience caused. Thank you.
Regards,
Cynthia.

On Sat, Mar 28, 2020 at 1:18 PM Cynthia Omovoiye <[hidden email]> wrote:

> Hi Zoran,
> Thank you for your feedback, I have been able to effect your corrections
> and I have been able to pull the Contributors data and have it display on
> the page but I am concerned that the *information[contributors3.jpg] *retrieved
> from this data do not correspond with the
> *information[contributors2.jpg] *already listed in the Contributors
> section on the team.md page. Is there somewhere else to pull this data from
> or can I go on with what I have? Thank you.
>
>
>
>
> With kind regards,
> Cynthia.
>
>
> On Thu, Mar 26, 2020, 10:09 AM Zoran Regvart <[hidden email]> wrote:
>
>> Hi Cynthia,
>> I've found that the contributors API[1] perhaps the contributors API
>> doesn't require authentication. Though we might end up hitting the
>> rate limits and then requiring the token. We can configure that on the
>> build server, for example we can provide the token via an environment
>> variable.
>>
>> What I think will be difficult is that with the responses from GitHub,
>> there is usually a limit on the number of results returned. So most
>> likely there will be a need for paginating the results. Not sure how
>> difficult that would be in Hugo.
>>
>> I noticed that there's a JSON data file in your changes, I would try
>> to avoid that if possible and fetch the data dynamically.
>>
>> It would be easier to have this conversation on a GitHub pull request,
>> so you might want to consider creating one for this.
>>
>> zoran
>>
>> [1] https://developer.github.com/v3/repos/#list-contributors
>>
>> On Wed, Mar 25, 2020 at 8:35 PM Cynthia Omovoiye <[hidden email]>
>> wrote:
>> >
>> > Hi Zoran,
>> > I have been able to pull all required data from the JSON files into the
>> team page for Committers as shown in the screenshot below(committers.png),
>> I have also been able to set the GitHub API that extracts the collaborators
>> data from the repository and rendered it to display on the team page.
>> However, it will require a GitHub access token from the Apache GitHub
>> account for the data to be displayed but I was able to generate one on my
>> forked repository of the Apache camel website for testing which returned
>> the data displayed in the screenshot below(contributors.png). Thus, if an
>> access token  generated by an admin of the Apache camel website repository
>> is inputted in the field I have made available for it in("
>> /layouts/shortcodes/contributors.html " {{$token :=" "}}), we would be able
>> get the data of the collaborators  displayed on the web page.
>> >
>> https://github.com/CynthiaOmovoiye/camel-website/tree/pullingDataFromJsonApis
>> >
>> > On Fri, Mar 20, 2020 at 9:44 AM Omar Al-Safi <[hidden email]> wrote:
>> >>
>> >> HI Cynthia,
>> >>
>> >> The organization names are being entered manually in the GitHub
>> repository
>> >> <
>> https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md
>> >,
>> >> perhaps you can scrap that page manually to extract this information?
>> >>
>> >> Regards,
>> >> Omar
>> >>
>> >> On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]>
>> >> wrote:
>> >>
>> >> > Hi Zoran,
>> >> >
>> >> > I have gone through the JSON files you provided and I would like to
>> know
>> >> > where I can find the organization names as displayed on the table on
>> the
>> >> > team page.
>> >> >
>> >> > Thank you,
>> >> > Cynthia.
>> >> >
>> >> > On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]>
>> wrote:
>> >> >
>> >> > > Hi Cynthia,
>> >> > > on the Camel website we have a page where we list all the team
>> members:
>> >> > >
>> >> > > https://camel.apache.org/community/team/
>> >> > >
>> >> > > I think we can get the data from that page from the
>> >> > > projects.apache.org, for example this page:
>> >> > >
>> >> > > https://projects.apache.org/committee.html?camel
>> >> > >
>> >> > > Lists all the PMC members (PMC Roster) and the Comitters, and I
>> think
>> >> > > the data is fetched from JSON at:
>> >> > >
>> >> > > https://projects.apache.org/json/foundation/committees.json
>> >> > >
>> >> > > There's a lot of data in JSON files here, that you can also take a
>> look
>> >> > at:
>> >> > >
>> >> > > https://projects.apache.org/json/
>> >> > >
>> >> > > We also have Contributors, folk much like you, that aren't
>> associated
>> >> > > with Apache Software Foundation but help out from time to time and
>> >> > > submit changes via GitHub. GitHub also offers this data as JSON,
>> here
>> >> > > is the documentation on how to fetch the list of all contributors:
>> >> > >
>> >> > >
>> https://developer.github.com/v3/repos/collaborators/#list-collaborators
>> >> > >
>> >> > > zoran
>> >> > >
>> >> > > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <
>> [hidden email]>
>> >> > > wrote:
>> >> > > >
>> >> > > > Hi,
>> >> > > > I am Cynthia Omovoiye, an applicant for Outreachy internship and
>> I
>> >> > picked
>> >> > > > interest in the Apache Camel site project with CAMEL 13894 as my
>> main
>> >> > > > focus.
>> >> > > >
>> >> > > > Although I am quite new to this topic, I am ready to learn and
>> >> > contribute
>> >> > > > to this project. Please, I would like to know what is expected
>> of me in
>> >> > > the
>> >> > > > completion of this project? I understand that I have to update
>> the
>> >> > > database
>> >> > > > of Contributors by pulling in their data through an API. I,
>> however,
>> >> > > don't
>> >> > > > know where I am required to pull it from and where I am supposed
>> to
>> >> > > upload
>> >> > > > it to.
>> >> > > >
>> >> > > > Please, assist me with this area. Thank you for your assistance.
>> >> > >
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Zoran Regvart
>> >> > >
>> >> >
>>
>>
>>
>> --
>> Zoran Regvart
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Cynthia Omovoiye
Hi Zoran,
 I have pushed the recent changes to the repository but I have a few
concerns which I have stated on the conversation.
Thank you.
Regards,
Cynthia.

On Sat, Mar 28, 2020 at 2:42 PM Cynthia Omovoiye <[hidden email]> wrote:

> Hi Zoran,
>
> Please disregard the previous email, I have been able to sort the issue
> out.
> Sorry for any inconvenience caused. Thank you.
> Regards,
> Cynthia.
>
> On Sat, Mar 28, 2020 at 1:18 PM Cynthia Omovoiye <[hidden email]>
> wrote:
>
>> Hi Zoran,
>> Thank you for your feedback, I have been able to effect your corrections
>> and I have been able to pull the Contributors data and have it display on
>> the page but I am concerned that the *information[contributors3.jpg] *retrieved
>> from this data do not correspond with the
>> *information[contributors2.jpg] *already listed in the Contributors
>> section on the team.md page. Is there somewhere else to pull this data from
>> or can I go on with what I have? Thank you.
>>
>>
>>
>>
>> With kind regards,
>> Cynthia.
>>
>>
>> On Thu, Mar 26, 2020, 10:09 AM Zoran Regvart <[hidden email]> wrote:
>>
>>> Hi Cynthia,
>>> I've found that the contributors API[1] perhaps the contributors API
>>> doesn't require authentication. Though we might end up hitting the
>>> rate limits and then requiring the token. We can configure that on the
>>> build server, for example we can provide the token via an environment
>>> variable.
>>>
>>> What I think will be difficult is that with the responses from GitHub,
>>> there is usually a limit on the number of results returned. So most
>>> likely there will be a need for paginating the results. Not sure how
>>> difficult that would be in Hugo.
>>>
>>> I noticed that there's a JSON data file in your changes, I would try
>>> to avoid that if possible and fetch the data dynamically.
>>>
>>> It would be easier to have this conversation on a GitHub pull request,
>>> so you might want to consider creating one for this.
>>>
>>> zoran
>>>
>>> [1] https://developer.github.com/v3/repos/#list-contributors
>>>
>>> On Wed, Mar 25, 2020 at 8:35 PM Cynthia Omovoiye <[hidden email]>
>>> wrote:
>>> >
>>> > Hi Zoran,
>>> > I have been able to pull all required data from the JSON files into
>>> the team page for Committers as shown in the screenshot
>>> below(committers.png), I have also been able to set the GitHub API that
>>> extracts the collaborators data from the repository and rendered it to
>>> display on the team page. However, it will require a GitHub access token
>>> from the Apache GitHub account for the data to be displayed but I was able
>>> to generate one on my forked repository of the Apache camel website for
>>> testing which returned the data displayed in the screenshot
>>> below(contributors.png). Thus, if an access token  generated by an admin of
>>> the Apache camel website repository is inputted in the field I have made
>>> available for it in(" /layouts/shortcodes/contributors.html " {{$token :="
>>> "}}), we would be able get the data of the collaborators  displayed on the
>>> web page.
>>> >
>>> https://github.com/CynthiaOmovoiye/camel-website/tree/pullingDataFromJsonApis
>>> >
>>> > On Fri, Mar 20, 2020 at 9:44 AM Omar Al-Safi <[hidden email]> wrote:
>>> >>
>>> >> HI Cynthia,
>>> >>
>>> >> The organization names are being entered manually in the GitHub
>>> repository
>>> >> <
>>> https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md
>>> >,
>>> >> perhaps you can scrap that page manually to extract this information?
>>> >>
>>> >> Regards,
>>> >> Omar
>>> >>
>>> >> On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]
>>> >
>>> >> wrote:
>>> >>
>>> >> > Hi Zoran,
>>> >> >
>>> >> > I have gone through the JSON files you provided and I would like to
>>> know
>>> >> > where I can find the organization names as displayed on the table
>>> on the
>>> >> > team page.
>>> >> >
>>> >> > Thank you,
>>> >> > Cynthia.
>>> >> >
>>> >> > On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]>
>>> wrote:
>>> >> >
>>> >> > > Hi Cynthia,
>>> >> > > on the Camel website we have a page where we list all the team
>>> members:
>>> >> > >
>>> >> > > https://camel.apache.org/community/team/
>>> >> > >
>>> >> > > I think we can get the data from that page from the
>>> >> > > projects.apache.org, for example this page:
>>> >> > >
>>> >> > > https://projects.apache.org/committee.html?camel
>>> >> > >
>>> >> > > Lists all the PMC members (PMC Roster) and the Comitters, and I
>>> think
>>> >> > > the data is fetched from JSON at:
>>> >> > >
>>> >> > > https://projects.apache.org/json/foundation/committees.json
>>> >> > >
>>> >> > > There's a lot of data in JSON files here, that you can also take
>>> a look
>>> >> > at:
>>> >> > >
>>> >> > > https://projects.apache.org/json/
>>> >> > >
>>> >> > > We also have Contributors, folk much like you, that aren't
>>> associated
>>> >> > > with Apache Software Foundation but help out from time to time and
>>> >> > > submit changes via GitHub. GitHub also offers this data as JSON,
>>> here
>>> >> > > is the documentation on how to fetch the list of all contributors:
>>> >> > >
>>> >> > >
>>> https://developer.github.com/v3/repos/collaborators/#list-collaborators
>>> >> > >
>>> >> > > zoran
>>> >> > >
>>> >> > > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <
>>> [hidden email]>
>>> >> > > wrote:
>>> >> > > >
>>> >> > > > Hi,
>>> >> > > > I am Cynthia Omovoiye, an applicant for Outreachy internship
>>> and I
>>> >> > picked
>>> >> > > > interest in the Apache Camel site project with CAMEL 13894 as
>>> my main
>>> >> > > > focus.
>>> >> > > >
>>> >> > > > Although I am quite new to this topic, I am ready to learn and
>>> >> > contribute
>>> >> > > > to this project. Please, I would like to know what is expected
>>> of me in
>>> >> > > the
>>> >> > > > completion of this project? I understand that I have to update
>>> the
>>> >> > > database
>>> >> > > > of Contributors by pulling in their data through an API. I,
>>> however,
>>> >> > > don't
>>> >> > > > know where I am required to pull it from and where I am
>>> supposed to
>>> >> > > upload
>>> >> > > > it to.
>>> >> > > >
>>> >> > > > Please, assist me with this area. Thank you for your assistance.
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > > --
>>> >> > > Zoran Regvart
>>> >> > >
>>> >> >
>>>
>>>
>>>
>>> --
>>> Zoran Regvart
>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: CAMEL 13894: Pulling Contributor data from JSON APIs

Zoran Regvart-2
Hi Cynthia,
yeah, I've been pondering about the need for the token and I guess the
Netlify build machines are always on the low end of GitHub limits.
I've noticed that there's also an issue with adding the GitHub token
to the query parameters -- it has been deprecated by GitHub and will
be removed later this year[1][2]. And we can't set HTTP headers for
requests done by Hugo[3].

What could be done is that we prepare the JSON file with the
contributors and feed that to Hugo. I'll have a look at that.

zoran

[1] https://developer.github.com/changes/2019-11-05-deprecated-passwords-and-authorizations-api/
[2] https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param/
[3] https://gohugo.io/templates/data-templates/#authentication-when-using-rest-urls

On Mon, Apr 6, 2020 at 8:11 PM Cynthia Omovoiye <[hidden email]> wrote:

>
> Hi Zoran,
>  I have pushed the recent changes to the repository but I have a few
> concerns which I have stated on the conversation.
> Thank you.
> Regards,
> Cynthia.
>
> On Sat, Mar 28, 2020 at 2:42 PM Cynthia Omovoiye <[hidden email]> wrote:
>
> > Hi Zoran,
> >
> > Please disregard the previous email, I have been able to sort the issue
> > out.
> > Sorry for any inconvenience caused. Thank you.
> > Regards,
> > Cynthia.
> >
> > On Sat, Mar 28, 2020 at 1:18 PM Cynthia Omovoiye <[hidden email]>
> > wrote:
> >
> >> Hi Zoran,
> >> Thank you for your feedback, I have been able to effect your corrections
> >> and I have been able to pull the Contributors data and have it display on
> >> the page but I am concerned that the *information[contributors3.jpg] *retrieved
> >> from this data do not correspond with the
> >> *information[contributors2.jpg] *already listed in the Contributors
> >> section on the team.md page. Is there somewhere else to pull this data from
> >> or can I go on with what I have? Thank you.
> >>
> >>
> >>
> >>
> >> With kind regards,
> >> Cynthia.
> >>
> >>
> >> On Thu, Mar 26, 2020, 10:09 AM Zoran Regvart <[hidden email]> wrote:
> >>
> >>> Hi Cynthia,
> >>> I've found that the contributors API[1] perhaps the contributors API
> >>> doesn't require authentication. Though we might end up hitting the
> >>> rate limits and then requiring the token. We can configure that on the
> >>> build server, for example we can provide the token via an environment
> >>> variable.
> >>>
> >>> What I think will be difficult is that with the responses from GitHub,
> >>> there is usually a limit on the number of results returned. So most
> >>> likely there will be a need for paginating the results. Not sure how
> >>> difficult that would be in Hugo.
> >>>
> >>> I noticed that there's a JSON data file in your changes, I would try
> >>> to avoid that if possible and fetch the data dynamically.
> >>>
> >>> It would be easier to have this conversation on a GitHub pull request,
> >>> so you might want to consider creating one for this.
> >>>
> >>> zoran
> >>>
> >>> [1] https://developer.github.com/v3/repos/#list-contributors
> >>>
> >>> On Wed, Mar 25, 2020 at 8:35 PM Cynthia Omovoiye <[hidden email]>
> >>> wrote:
> >>> >
> >>> > Hi Zoran,
> >>> > I have been able to pull all required data from the JSON files into
> >>> the team page for Committers as shown in the screenshot
> >>> below(committers.png), I have also been able to set the GitHub API that
> >>> extracts the collaborators data from the repository and rendered it to
> >>> display on the team page. However, it will require a GitHub access token
> >>> from the Apache GitHub account for the data to be displayed but I was able
> >>> to generate one on my forked repository of the Apache camel website for
> >>> testing which returned the data displayed in the screenshot
> >>> below(contributors.png). Thus, if an access token  generated by an admin of
> >>> the Apache camel website repository is inputted in the field I have made
> >>> available for it in(" /layouts/shortcodes/contributors.html " {{$token :="
> >>> "}}), we would be able get the data of the collaborators  displayed on the
> >>> web page.
> >>> >
> >>> https://github.com/CynthiaOmovoiye/camel-website/tree/pullingDataFromJsonApis
> >>> >
> >>> > On Fri, Mar 20, 2020 at 9:44 AM Omar Al-Safi <[hidden email]> wrote:
> >>> >>
> >>> >> HI Cynthia,
> >>> >>
> >>> >> The organization names are being entered manually in the GitHub
> >>> repository
> >>> >> <
> >>> https://raw.githubusercontent.com/apache/camel-website/master/content/community/team.md
> >>> >,
> >>> >> perhaps you can scrap that page manually to extract this information?
> >>> >>
> >>> >> Regards,
> >>> >> Omar
> >>> >>
> >>> >> On Thu, Mar 19, 2020 at 11:15 PM Cynthia Omovoiye <[hidden email]
> >>> >
> >>> >> wrote:
> >>> >>
> >>> >> > Hi Zoran,
> >>> >> >
> >>> >> > I have gone through the JSON files you provided and I would like to
> >>> know
> >>> >> > where I can find the organization names as displayed on the table
> >>> on the
> >>> >> > team page.
> >>> >> >
> >>> >> > Thank you,
> >>> >> > Cynthia.
> >>> >> >
> >>> >> > On Mon, Mar 9, 2020, 11:39 AM Zoran Regvart <[hidden email]>
> >>> wrote:
> >>> >> >
> >>> >> > > Hi Cynthia,
> >>> >> > > on the Camel website we have a page where we list all the team
> >>> members:
> >>> >> > >
> >>> >> > > https://camel.apache.org/community/team/
> >>> >> > >
> >>> >> > > I think we can get the data from that page from the
> >>> >> > > projects.apache.org, for example this page:
> >>> >> > >
> >>> >> > > https://projects.apache.org/committee.html?camel
> >>> >> > >
> >>> >> > > Lists all the PMC members (PMC Roster) and the Comitters, and I
> >>> think
> >>> >> > > the data is fetched from JSON at:
> >>> >> > >
> >>> >> > > https://projects.apache.org/json/foundation/committees.json
> >>> >> > >
> >>> >> > > There's a lot of data in JSON files here, that you can also take
> >>> a look
> >>> >> > at:
> >>> >> > >
> >>> >> > > https://projects.apache.org/json/
> >>> >> > >
> >>> >> > > We also have Contributors, folk much like you, that aren't
> >>> associated
> >>> >> > > with Apache Software Foundation but help out from time to time and
> >>> >> > > submit changes via GitHub. GitHub also offers this data as JSON,
> >>> here
> >>> >> > > is the documentation on how to fetch the list of all contributors:
> >>> >> > >
> >>> >> > >
> >>> https://developer.github.com/v3/repos/collaborators/#list-collaborators
> >>> >> > >
> >>> >> > > zoran
> >>> >> > >
> >>> >> > > On Sun, Mar 8, 2020 at 7:19 AM Cynthia Omovoiye <
> >>> [hidden email]>
> >>> >> > > wrote:
> >>> >> > > >
> >>> >> > > > Hi,
> >>> >> > > > I am Cynthia Omovoiye, an applicant for Outreachy internship
> >>> and I
> >>> >> > picked
> >>> >> > > > interest in the Apache Camel site project with CAMEL 13894 as
> >>> my main
> >>> >> > > > focus.
> >>> >> > > >
> >>> >> > > > Although I am quite new to this topic, I am ready to learn and
> >>> >> > contribute
> >>> >> > > > to this project. Please, I would like to know what is expected
> >>> of me in
> >>> >> > > the
> >>> >> > > > completion of this project? I understand that I have to update
> >>> the
> >>> >> > > database
> >>> >> > > > of Contributors by pulling in their data through an API. I,
> >>> however,
> >>> >> > > don't
> >>> >> > > > know where I am required to pull it from and where I am
> >>> supposed to
> >>> >> > > upload
> >>> >> > > > it to.
> >>> >> > > >
> >>> >> > > > Please, assist me with this area. Thank you for your assistance.
> >>> >> > >
> >>> >> > >
> >>> >> > >
> >>> >> > > --
> >>> >> > > Zoran Regvart
> >>> >> > >
> >>> >> >
> >>>
> >>>
> >>>
> >>> --
> >>> Zoran Regvart
> >>>
> >>



--
Zoran Regvart