Design a site like this with
Get started

Searching for knowledge/tips on building web scrapers/crawlers using Chrome Manifest V3 extensions


Didn't wanna subscribe and crosspost.

Hi Reddit. Hoping to gather some advice from the programmers on this subreddit.

Last semester, me and a team of my academic cohorts were tasked by a non-profit organization to build a web scraper/crawler as a chrome extension.

We tried… and tried… but failed because me and my team just couldn't figure out how to do it! Our client wanted us to save videos, images and web pages offline. The client wanted all of those to look just as they appeared when you first navigated to them.

And as I'm sure many of you are aware, it's hard to do such a thing given that media storage on websites isn't standardized. The only thing we gave the client at the end of the semester were reports on our failures.

Anyway, a new team is heading the project now. We didn't have much to give them in terms of tips. Do any of you have some advice on this? Or perhaps an example of an existing solution to this problem? I'd like to help nudge them in the right direction.

Thank you!

submitted by /u/FlashDaggerX
[link] [comments]

from Software Development – methodologies, techniques, and tools. Covering Agile, RUP, Waterfall + more!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this:
search previous next tag category expand menu location phone mail time cart zoom edit close