[Data Science Foundations Project] Seeking Volunteers!

[Salkimmich]

Come join us, and invite your (data science) friends:

We are seeking:

1. Help with Automated link extraction* for blogs, mailing lists, and community portals (and making sure it is accurate) is where we need developer help now
2. Help us run an online/live in London Hackathon on this data - October 30th (or another convenient date).

You can give this repo for a full review, and reply to this Issues thread for best ways to get started!

https://github.com/chaoss/wg-data-science/tree/main/dataset/foundational-stats-research

[PredictiveManish]

I can help for online!

[Salkimmich]

@PredictiveManish thank you so much for joining in, we'd love your contribution on this! Here's some context and the next steps I'd recommend:

Why This Work Matters

Completing this Apache metadata is one of the remaining blockers for two major outcomes:

1. Creating a reproducible CHAOSS-aligned template for foundations
   By establishing a complete, validated metadata structure for Apache, CNCF, and Eclipse, we can produce a reference template that other foundations can follow when preparing for CHAOSS reviews. Your work helps define what strong metadata hygiene looks like in practice and provides a model that other ecosystems can adopt.

2. Enabling reliable prediction models across all three ecosystems
   Our statistical and engagement prediction models rely on consistent metadata across foundations. Missing communication fields in Apache currently prevent us from running analyses uniformly. Your manual validation work allows us to complete cross-foundation modeling, benchmarking, and outlier detection with confidence.

This really helps, so thank you for volunteering to help with the Apache Foundation manual metadata review!

How to Get Started

1. Review the Apache project list

We are collecting communication metadata for Apache projects, including Slack, Discord, Zulip, IRC, Mailing Lists, Forums, and Blogs.

Browse projects here:
https://projects.apache.org/projects.html

Select 5–10 projects to start. For each project, open the project homepage or GitHub repository and look for any publicly listed communication or community links.

2. Record what you find

Please create a JSON or CSV file with one record per project. Use the following structure:

{
  "project": "apache-airflow",
  "slack_url": "https://apache-airflow.slack.com",
  "mailing_list": "https://lists.apache.org/list.html?users@airflow.apache.org",
  "discord_url": null,
  "zulip_url": null,
  "forum_url": null
}

If no public communication channels are found, mark the fields as null.

3. Submit your contribution

Once you have your first batch:

1. Fork the repository (you can do that now, if you haven't already)
2. Add your file under:
   dataset/foundational-stats-research/data/processed/apache_manual_reviews/
3. Open a Pull Request titled something like:
   “Added manual communication metadata for 10 Apache projects”

If you need help with GitHub or the pull request workflow, comment here and we will assist, and ready to answer any questions here.

There's a lot of support for this project through our broader community, which you can join through any of the links outlined here: https://chaoss.community/inside-the-chaoss-data-science-working-group/

Project Next Steps

We can help you to build a small validation notebook for completeness scoring or help you experiment with automated link extraction, or you can join in on the data analysis review (or both)! If you have great ideas, we're here to help make them happen.

Looking forward to the magic we can unlock with a few JSON files soon!

[HimasreeKolathur24]

Hi 👋
I’d like to work on this issue. I’ll start with a small batch (5–10 Apache projects) and submit a PR with manual communication metadata.

Please let me know if that sounds good.

[oree-xx]

@Salkimmich I am interested in volunteering for this!

[E-STAT]

Hi @HimasreeKolathur24

Thank you so much for showing interest in working on this too!

Surely, it sounds good and we'd be expecting your PR!

[HimasreeKolathur24]

Hi @Salkimmich and @E-STAT,

Thank you both! I’ve now submitted my first PR for this issue:
#167

Looking forward to your feedback — thanks!

[nikitagrover19]

Hi @Salkimmich and @E-STAT

Thanks for the clear guidance on this issue. I’d like to contribute as well by adding manually validated communication metadata for Apache projects, following the current schema and directory structure used in the repo.

I’ll make sure to work on projects that aren’t already covered and will submit a PR once ready.

Please let me know if there’s anything specific you’d like me to focus on.

Thanks!

[E-STAT]

Hi @nikitagrover19

It's exciting to have you contribute to this!

There is an open PR here #173 that contains the tracker that can help us all to better manage the projects and avoid duplicated efforts.

You should follow that tracker and the "IMPORTANT INFO" section of the readme. And I'd be expecting your PR!

[nikitagrover19]

Hi @E-STAT, thanks for setting up the tracker and schema in #173 — this really clarifies the workflow.

Since the data/raw/apache.json file is being introduced in that PR and doesn’t exist on main yet, I wanted to confirm the preferred approach before proceeding.

Would you like contributors to base their Apache metadata PRs on the branch from #173 (i.e. stack on top of it), or should we wait until #173 is merged and then open PRs against main?

I’m ready to proceed either way — just wanted to align before committing changes.
Thanks!

[E-STAT]

I'd say we wait for it to be merged. I've prompoted @Salkimmich and @geekygirldawn on Slack.

Hopefully, they'd have some time to check.

Thanks!