1- __ __ _ __ ____
2- /\ \/\ \/\`'__\/',__\
3- \ \ \_\ \ \ \//\__, `\
4- \ \____/\ \_\\/\____/
5- \/___/ \/_/ \/___/
1+ ```
2+ __ __ _ __ ____
3+ /\ \/\ \/\`'__\/',__\
4+ \ \ \_\ \ \ \//\__, `\
5+ \ \____/\ \_\\/\____/
6+ \/___/ \/_/ \/___/
7+ ```
68
79> ** U** niversal ** R** eddit ** S** craper - A comprehensive Reddit scraping command-line tool written in Python.
810
1315![ Total lines] ( https://img.shields.io/tokei/lines/github/JosephLai241/URS )
1416![ License] ( https://img.shields.io/github/license/JosephLai241/URS )
1517
16- ```
17- [-h]
18- [-e]
19- [-v]
18+ # Sponsors
2019
21- [-t [<optional_date>]]
22- [--check]
20+ <p align =" center " >
21+ <a href =" https://dashboard.thordata.com/register?invitation_code=8I13V2C7 " >
22+ <img src="https://github.com/user-attachments/assets/67052ea0-d05c-4fd5-998c-f819fd233a8a" />
23+ </a >
24+ </p >
2325
24- [-r <subreddit> <(h|n|c|t|r|s)> <n_results_or_keywords> [<optional_time_filter>]]
25- [-y]
26- [--csv]
27- [--rules]
28- [-u <redditor> <n_results>]
29- [-c <submission_url> <n_results>]
30- [--raw]
31- [-b]
32- [--csv]
26+ <p align =" center " style =" max-width : 500px ; margin : auto ;" >
27+ <strong >Thordata</strong >’s tools are particularly useful in scenarios that require large-scale web scraping through their
28+ <a href =" https://www.thordata.com/products/web-scraper/?ls=EDBORvrR&lk=wb " >Web Scraper API</a >
29+ , API-based data extraction, or reliable
30+ <a href =" https://www.thordata.com/products/residential-proxies/?ls=EDBORvrR&lk=wb " >Proxy</a >
31+ infrastructure.
32+ If you plan to use Thordata's tools, you can support the project via this <a href =" https://dashboard.thordata.com/register?invitation_code=8I13V2C7 " >affiliate link</a >.
33+ </p >
3334
34- [-lr <subreddit>]
35- [-lu <redditor>]
35+ ## Previous Sponsors
3636
37- [--nosave]
38- [--stream-submissions]
39-
40- [-f <file_path>]
41- [--csv]
42- [-wc <file_path> [<optional_export_format>]]
43- [--nosave]
44- ```
37+ - [ lolfilmworks]
4538
4639# Table of Contents
4740
4841- [ Contact] ( #contact )
4942- [ Introduction] ( #introduction )
43+ - [ Usage Overview] ( #usage-overview )
5044- [ "Where’s the Manual?"] ( #wheres-the-manual )
5145 - [ ` URS ` Manual] ( #urs-manual )
5246- [ Demo GIFs] ( #demo-gifs )
5852 - [ Generating Wordclouds] ( #generating-wordclouds )
5953 - [ Checking PRAW Rate Limits] ( #checking-praw-rate-limits )
6054 - [ Displaying Directory Tree] ( #displaying-directory-tree )
61- - [ Sponsors] ( #sponsors )
6255
6356# Contact
6457
@@ -83,11 +76,43 @@ This is a comprehensive Reddit scraping tool that integrates multiple features:
8376 - Generate frequencies for words that are found in submission titles, bodies, and/or comments
8477 - Generate a wordcloud from scrape results
8578
79+ # Usage Overview
80+
81+ ```
82+ [-h]
83+ [-e]
84+ [-v]
85+
86+ [-t [<optional_date>]]
87+ [--check]
88+
89+ [-r <subreddit> <(h|n|c|t|r|s)> <n_results_or_keywords> [<optional_time_filter>]]
90+ [-y]
91+ [--csv]
92+ [--rules]
93+ [-u <redditor> <n_results>]
94+ [-c <submission_url> <n_results>]
95+ [--raw]
96+ [-b]
97+ [--csv]
98+
99+ [-lr <subreddit>]
100+ [-lu <redditor>]
101+
102+ [--nosave]
103+ [--stream-submissions]
104+
105+ [-f <file_path>]
106+ [--csv]
107+ [-wc <file_path> [<optional_export_format>]]
108+ [--nosave]
109+ ```
110+
86111# "Where’s the Manual?"
87112
88113### [ ` URS ` Manual] [ urs manual ]
89114
90- This ` README ` has become too long to comfortably contain all usage information for this tool. Consequently, the information that used to be in this file has been moved to a separate manual created with [ mdBook] [ mdbook ] , a Rust command-line tool for creating books from Markdown files.
115+ This ` README ` has become too long to comfortably contain all usage information for this tool. Consequently, the information that used to be in this file has been moved to a separate manual created with [ mdBook] , a Rust command-line tool for creating books from Markdown files.
91116
92117> ** _ Note:_ ** You can also find the link in the About sidebar in this repository.
93118
@@ -99,78 +124,58 @@ Here are all the demo GIFs recorded for `URS`.
99124
100125## [ Subreddit Scraping] [ subreddit scraping manual link ]
101126
102- ![ subreddit demo] [ subreddit demo ]
127+ ![ subreddit demo]
103128
104129## [ Redditor Scraping] [ redditor scraping manual link ]
105130
106- ![ redditor demo] [ redditor demo ]
131+ ![ redditor demo]
107132
108133## [ Submission Comments Scraping] [ submission comments scraping manual link ]
109134
110- ![ submission comments demo] [ submission comments demo ]
135+ ![ submission comments demo]
111136
112137## [ Livestreaming Reddit] [ livestream scraping manual link ]
113138
114- ![ livestream subreddit demo] [ livestream subreddit demo ]
139+ ![ livestream subreddit demo]
115140
116141## [ Generating Word Frequencies] [ frequencies scraping manual link ]
117142
118- ![ frequencies demo] [ frequencies demo ]
143+ ![ frequencies demo]
119144
120145## [ Generating Wordclouds] [ wordcloud scraping manual link ]
121146
122- ![ wordcloud demo] [ wordcloud demo ]
147+ ![ wordcloud demo]
123148
124149## [ Checking PRAW Rate Limits] [ check praw rate limits manual link ]
125150
126- ![ check praw rate limits demo] [ check praw rate limits demo ]
151+ ![ check praw rate limits demo]
127152
128153## [ Displaying Directory Tree] [ display directory tree manual link ]
129154
130- ![ display directory tree demo] [ display directory tree demo ]
131-
132- # Sponsors
133-
134- This is a shout-out section for my patrons - thank you so much for sponsoring this project!
135-
136- - [ lolfilmworks] [ lolfilmworks ]
137-
138- <!-- Manual links-->
155+ ![ display directory tree demo]
139156
157+ [ check praw rate limits demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/utilities/check_rate_limit_demo.gif
140158[ check praw rate limits manual link ] : https://josephlai241.github.io/URS/utilities/rate-limit-checking.html
159+ [ codecov ] : https://codecov.io/gh/JosephLai241/URS
141160[ contributing manual link ] : https://josephlai241.github.io/URS/contributing/before-making-pull-or-feature-requests.html
161+ [ display directory tree demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/utilities/tree_demo.gif
142162[ display directory tree manual link ] : https://josephlai241.github.io/URS/utilities/tree.html
163+ [ frequencies demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/analytical_tools/frequencies_generator_demo.gif
143164[ frequencies scraping manual link ] : https://josephlai241.github.io/URS/analytical-tools/frequencies-and-wordclouds.html#generating-word-frequencies
144165[ livestream scraping manual link ] : https://josephlai241.github.io/URS/livestreaming-reddit/general-information.html
145- [ redditor scraping manual link ] : https://josephlai241.github.io/URS/scraping-reddit/redditor.html
146- [ submission comments scraping manual link ] : https://josephlai241.github.io/URS/scraping-reddit/submission-comments.html
147- [ subreddit scraping manual link ] : https://josephlai241.github.io/URS/scraping-reddit/subreddit.html
148- [ urs manual ] : https://josephlai241.github.io/URS
149- [ wordcloud scraping manual link ] : https://josephlai241.github.io/URS/analytical-tools/frequencies-and-wordclouds.html#generating-wordclouds
150-
151- <!-- PRAW SCRAPER DEMO GIFS -->
152-
153- [ check praw rate limits demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/utilities/check_rate_limit_demo.gif
154- [ display directory tree demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/utilities/tree_demo.gif
155- [ frequencies demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/analytical_tools/frequencies_generator_demo.gif
156166[ livestream subreddit demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/praw_scrapers/live_scrapers/livestream_subreddit_demo.gif
157- [ redditor demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/praw_scrapers/static_scrapers/Redditor_demo.gif
158- [ submission comments demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/praw_scrapers/static_scrapers/submission_comments_demo.gif
159- [ subreddit demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/praw_scrapers/static_scrapers/Subreddit_demo.gif
160- [ wordcloud demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/analytical_tools/wordcloud_generator_demo.gif
161-
162- <!-- Contact links-->
163-
164- [ urs project email ] : mailto:urs_project@protonmail.com
165-
166- <!-- Miscellaneous links-->
167-
168- [ codecov ] : https://codecov.io/gh/JosephLai241/URS
167+ [ lolfilmworks ] : https://github.com/lolfilmworks
169168[ mdbook ] : https://github.com/rust-lang/mdBook
170169[ nomad ] : https://github.com/JosephLai241/nomad
171170[ praw ] : https://praw.readthedocs.io/en/stable/
171+ [ redditor demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/praw_scrapers/static_scrapers/Redditor_demo.gif
172+ [ redditor scraping manual link ] : https://josephlai241.github.io/URS/scraping-reddit/redditor.html
172173[ releases ] : https://github.com/JosephLai241/URS/releases
173-
174- <!-- Sponsors links-->
175-
176- [ lolfilmworks ] : https://github.com/lolfilmworks
174+ [ submission comments demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/praw_scrapers/static_scrapers/submission_comments_demo.gif
175+ [ submission comments scraping manual link ] : https://josephlai241.github.io/URS/scraping-reddit/submission-comments.html
176+ [ subreddit demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/praw_scrapers/static_scrapers/Subreddit_demo.gif
177+ [ subreddit scraping manual link ] : https://josephlai241.github.io/URS/scraping-reddit/subreddit.html
178+ [ urs manual ] : https://josephlai241.github.io/URS
179+ [ urs project email ] : mailto:urs_project@protonmail.com
180+ [ wordcloud demo ] : https://github.com/JosephLai241/URS/blob/demo-gifs/analytical_tools/wordcloud_generator_demo.gif
181+ [ wordcloud scraping manual link ] : https://josephlai241.github.io/URS/analytical-tools/frequencies-and-wordclouds.html#generating-wordclouds
0 commit comments