Idiot-Proof Methods of Datahoarding
Dec. 22nd, 2024 03:47 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
bedes posting in ![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}
datahoardersA lot of datahoarding resources assume that you're intimately familiar with command prompts, webcrawlers, scripts, and more. But what if... you're stupid? What if you're extremely unfamiliar with those sorts of things, looking to have your hand held through the entire process? What if your name is Azure and you have an account called bedes on Dreamwidth.org? (Wait, that's getting too specific.)
Well, these resources are for you! Idiot-proof resources I've gathered for archiving stuff online! As tested and approved by an actual idiot!
(Quick disclaimer: I call myself "stupid" and "an idiot" lightheartedly, and as a morally neutral descriptor. Being stupid about certain things isn't bad! Which is why I'm making this post!)
Cobalt.tools: this is a great all-in-one resource for saving videos, audio, photos and gifs, with a large list of websites it can rip from. This list includes, but is not limited to, Facebook, Instagram, YouTube, Tumblr, Dailymotion, Pinterest, Reddit, and Twitch. It's open-source, easy to use, privacy-focused, ad-free, fast, and, frankly, It Just Works. No installation is required to use this tool.
Spotdownloader: a tool for ripping from Spotify with the highest possible quality! Very easy to use, and supports downloading a single song, a playlist, and an album, with all the important metadata about the song(s) in tact. (If downloading multiple songs, it puts them all in a convenient ZIP file.) Ad-free, and even has a userscript alternative. No installations are needed.
DiscordChatExporter: used to export any Discord message history to a file! It can export to HTML (dark/light), TXT, CSV and JSON, and supports Discord's form of markdown, embeds, attachments, and emojis. You have to install it via GitHub, and it has an excellent optional user interface that makes the process easy to follow!
Archive.org: known as the quintessential internet archive. I find it difficult to navigate what's already there sometimes, but, in terms of uploading your stuff, it's pretty easy! You do need an account to archive files, but archive.org is definitely the most useful for archiving webpages, which can be done without an account. There is also an Internet Archive extension, which you can use to save webpages quickly and easily, without needing to leave the page or open a new tab. You can also easily archive webpages via GhostArchive and Archive.Today, since backups are always necessary! No installations or downloads are required for any of these (except the Internet Archive extension, which requires you to add the extension).
Imgbrd-Grabber: a customizable tool for bulk-downloading images from imageboards, including (but not limited to) danbooru, safebooru, ArtStation, DeviantArt, Newgrounds and Pixiv, as well as many boorus specializing in pornographic content. The main reason that this tool is idiot-proof is thanks to the extremely thorough instructions provided for the installation and usage process.
Hydrus Network: you're gonna need a way to sort through all those new images you just downloaded, huh? Hydrus Network allows you to do just that -- making a locally-hosted booru of all of your art, with tags! It also supports bulk-downloading, like Imgbrd-Grabber, and shares a lot of the same supported sources, but has the notable unique features of being able to grab from Tumblr, and the ability to 'subscribe' to any gallery, repeating it every few days to keep up with new results. Also like Imgbrd-Grabber, it is an install which is mainly here thanks to its very thorough instructions, which walk the reader through everything.
AO3 Downloader: a life-saver for any person who has thought, "God, I wish I could download all of my bookmarks, but that would take sooo long to do individually." Another Github download which is saved by its thorough instructions!
tumblr-utils: a fantastic method for backing up your tumblr account. It's quite a pain to download and set up, but (say it with me, everyone!) it's saved with a Google Doc of extremely thorough instructions (skip to the "Tumblr-Utils" section... or read all of it! it's a great doc that goes over all sorts of different options for backing up your blog -- this is just the one I prefer). Once you get through setting it all up the first time, though, it works like a dream, with very simple command prompts, which are explained in the doc in layman's terms. These command prompts can save video and audio locally, fallback to the Internet Archive, save your likes, make an index of tags, and support ✨ incremental backups ✨! (Meaning that it continues from the last backup, instead of downloading the entire blog from scratch each time.)
If you know of any data-hoarding / archival resources that wasn't mentioned here, and you think even a total Python-illiterate doofus could get working, link it in the comments below! (Also, please include if it involves or requires any downloads, just because I think that's useful info.)
Well, these resources are for you! Idiot-proof resources I've gathered for archiving stuff online! As tested and approved by an actual idiot!
(Quick disclaimer: I call myself "stupid" and "an idiot" lightheartedly, and as a morally neutral descriptor. Being stupid about certain things isn't bad! Which is why I'm making this post!)
Cobalt.tools: this is a great all-in-one resource for saving videos, audio, photos and gifs, with a large list of websites it can rip from. This list includes, but is not limited to, Facebook, Instagram, YouTube, Tumblr, Dailymotion, Pinterest, Reddit, and Twitch. It's open-source, easy to use, privacy-focused, ad-free, fast, and, frankly, It Just Works. No installation is required to use this tool.
Spotdownloader: a tool for ripping from Spotify with the highest possible quality! Very easy to use, and supports downloading a single song, a playlist, and an album, with all the important metadata about the song(s) in tact. (If downloading multiple songs, it puts them all in a convenient ZIP file.) Ad-free, and even has a userscript alternative. No installations are needed.
DiscordChatExporter: used to export any Discord message history to a file! It can export to HTML (dark/light), TXT, CSV and JSON, and supports Discord's form of markdown, embeds, attachments, and emojis. You have to install it via GitHub, and it has an excellent optional user interface that makes the process easy to follow!
Archive.org: known as the quintessential internet archive. I find it difficult to navigate what's already there sometimes, but, in terms of uploading your stuff, it's pretty easy! You do need an account to archive files, but archive.org is definitely the most useful for archiving webpages, which can be done without an account. There is also an Internet Archive extension, which you can use to save webpages quickly and easily, without needing to leave the page or open a new tab. You can also easily archive webpages via GhostArchive and Archive.Today, since backups are always necessary! No installations or downloads are required for any of these (except the Internet Archive extension, which requires you to add the extension).
Imgbrd-Grabber: a customizable tool for bulk-downloading images from imageboards, including (but not limited to) danbooru, safebooru, ArtStation, DeviantArt, Newgrounds and Pixiv, as well as many boorus specializing in pornographic content. The main reason that this tool is idiot-proof is thanks to the extremely thorough instructions provided for the installation and usage process.
Hydrus Network: you're gonna need a way to sort through all those new images you just downloaded, huh? Hydrus Network allows you to do just that -- making a locally-hosted booru of all of your art, with tags! It also supports bulk-downloading, like Imgbrd-Grabber, and shares a lot of the same supported sources, but has the notable unique features of being able to grab from Tumblr, and the ability to 'subscribe' to any gallery, repeating it every few days to keep up with new results. Also like Imgbrd-Grabber, it is an install which is mainly here thanks to its very thorough instructions, which walk the reader through everything.
AO3 Downloader: a life-saver for any person who has thought, "God, I wish I could download all of my bookmarks, but that would take sooo long to do individually." Another Github download which is saved by its thorough instructions!
tumblr-utils: a fantastic method for backing up your tumblr account. It's quite a pain to download and set up, but (say it with me, everyone!) it's saved with a Google Doc of extremely thorough instructions (skip to the "Tumblr-Utils" section... or read all of it! it's a great doc that goes over all sorts of different options for backing up your blog -- this is just the one I prefer). Once you get through setting it all up the first time, though, it works like a dream, with very simple command prompts, which are explained in the doc in layman's terms. These command prompts can save video and audio locally, fallback to the Internet Archive, save your likes, make an index of tags, and support ✨ incremental backups ✨! (Meaning that it continues from the last backup, instead of downloading the entire blog from scratch each time.)
If you know of any data-hoarding / archival resources that wasn't mentioned here, and you think even a total Python-illiterate doofus could get working, link it in the comments below! (Also, please include if it involves or requires any downloads, just because I think that's useful info.)
