Entry tags:

[sticky entry] Sticky: Welcome to the Data Hoarding community!

This is a community for the hobby of data hoarding, and those interested in data hoarding.

Some resource links

Read more... )

Make an intro post if you'd like! Feel free to alter the template. ^.^


How to sort and label large image libraries?

Hi, I've been hoarding images and other data for a while but this is the first time I've gotten around to really trying to organize them and am looking for some help. I've been working on a library/archive/time capsule project that includes a large amount of images and I feel like some of them need context to understand and want to preserve that information for possible future users. I have a Mac computer so I've started labeling things with tags but I understand that this feature doesn't transfer across file systems. While poking around I found that .jpegs have a comments field in their metadata but this isn't reliably preserved, even on the same Mac it seems -- if I put a comment on a .jpeg on a flash drive, disconnect the drive and reattach it, I can't see it anymore. It seems like the only reliable method of attaching data would be through the directory tree and file name, and maybe a text document in the folder explaining some things that need more explanation cross referencing by the file name? Are there any methods I'm missing that aren't too fragile/unreliable/non-transferrable? Thanks.
tozka: title character sitting with a friend (Default)
[personal profile] tozka2025-03-18 10:48 am
Entry tags:

how to get RSS feeds for various websites

I recently posted about switching to using RSS feeds even more than I had been, and it was suggested to post it here as you can use RSS feeds to download a variety of things. For instance, if you want to download podcasts (perhaps using gPodder) you basically NEED RSS feed so you can automatically grab episodes instead of trying to rip them one by one.

Here is my original post. A slightly shortened version follows:

If you've never used RSS before, here's a good intro guide from Melonland.

Feed readers


I use Feedbro (review here) which is a browser-based extension that works in Firefox (and Firefox forks), Chrome and Edge. I don't need to read my feeds on my phone, so having it on the desktop only is fine for me.

Other options are: NetNewsWire (iOS), Dreamwidth's feed thing

How to find RSS feeds


All blogs/websites have an RSS feed somewhere, even if it's not visible-- try adding /rss or /feed to the end of the blog URL to find it. But there's actually a surprising amount of other things you can get an RSS feed for!

Here's what I've transferred to my feedreader:

1. Tumblr
Absolutely fantastic because the endless scroll is gone, there's no ads, and I see EVERY UPDATE chronologically. Feeds can be found by adding /rss to the end of the URL: [user].tumblr.com/rss

2. Reddit
Another banger because I can see EVERY new post now; and I don't even have to join a subreddit to see them which is great. Add /.rss to the end of the URL: https://www.reddit.com/r/[subreddit]/.rss

3. Youtube channels
Yet another great idea because I see every new video from every channel now, instead of YT pushing up only 3 or 4 of them on my homescreen. I cheated and used Feedbro to automatically find the RSS feed but it looks something like this: https://www.youtube.com/feeds/videos.xml?channel_id=[ID]

4. Podcasts
I actually use Overcast to subscribe to podcasts BUT you can definitely do it in a feedreader. Most podcast websites have an RSS feed somewhere; even Spotify podcasts have them, because they've enabled an RSS feed so, like, Apple Podcasts can scrape it or whatever. Paste the Spotify URL into GetRSSFeed.com to find it.

5. Substack
I hate Substack but they've been pushing for more people to make newsletters over there lately, so a lot of people have opened new accounts. Ugh. But still, there's some good writers over there, and I want to read their stuff. Add /feed at the end of the URL: https://[user].substack.com/feed

6. Newsletters
I get overwhelmed if there's too many emails in my inbox (especially from strangers selling me stuff), so I'm converting whatever I can to an RSS feed using Kill the Newsletter, which basically gives you an email to use to sign up to newsletters and then it converts it to a feed for you! It's open source, so you can host it yourself if you'd like.

Caveat: I haven't tried to subscribe to a private feed, but most feedreaders have an option to add in a username/password to access those. However, I don't know how it'd work for sites that are more like memberships, like Youtube's community thing.

Not possible to RSS because the company that made them sucks: Facebook, Instagram, Threads (so any Meta app), TikTok

Possible to RSS but I haven't done it: Bluesky, Flickr, Mastodon, Medium, AO3, Dreamwidth

Two interesting articles about Media Preservation

Both of these articles were shared by The Rec Center mailing list.


Someone Has To Save The Film And TV That Studios Won’t

Preservation is an act of collective willpower and effort. It cannot just be one person. I don’t want to live in a world where the things I love actually disappear.

This interview was really cool! It's about a guy who learned how to rip really high-quality digital video from from VHS tapes. What an amazing skill to have! I was surprisingly infuriated by the idea that someone had altered the ending of a wrestling match from 1992; I don't give a shit about wrestling but it offends my love of the truth.

The Digital Packrat Manifesto

This article probably won't be anything that data hoarders haven't already thought of, although it did bring up some aspects of enshittification (especially of Spotify) that I hadn't heard about. Still a good read, and maybe a good share for those who might not be data hoarders yet if you want to try to get them to come around to the idea.
tozka: woman typing onto a very old computer (computer black and white)
[personal profile] tozka2025-03-08 03:08 pm

how to build a trans literature microlibrary

I stumbled across a series of posts about building a trans literature microlibrary, which includes a WHOLE BUNCH of detailed info about what to download, how to download it, where to keep your downloads, how to do backups, and so on. The first post also has guidelines and advice for a wide range of people, including academics, librarians, booksellers, etc.

It's really good info, especially if you've never considered saving anything for offline use or why you might want to in the first place.

Here's the posts in order:
1. The Trans Literature Preservation Project: A Practical Guide to Resisting Censorship
1b. There's also a printable zine version (PDF)
2. How to Build Your Trans Microlibrary
3. Archiving Your Trans Microlibrary, Part One
4. Hardware and Storage: Archiving Your Trans Microlibrary, Part Two
volkameria: Rodney (SGA) talking in front of a computer screen (pic#rodney_talking)
[personal profile] volkameria2025-02-27 05:43 pm

Resource Aggregates and Masterposts

Credit goes to [personal profile] ysabetwordsmith for bringing this up! One way to archive that has been a blindspot for me is looking at previous aggregates - link clouds and master posts on various topics. To standardize the language around these, I'll be referring to them as masterposts from here on.

A masterpost is a collection of resources and links categorized by one person or multiple people as a primer on a certain subject. You may have seen these on Reddit or Tumblr (especially for study guides, tips, and college advice on Tumblr, there were lots of these.)
Masterposts can provide valuable insight into what is considered generally important for a given topic, and are a great bouncing off place if you want to start learning about something but aren't sure where to begin!

Some examples of masterposts:

Survival Books, Manuals, and Prepper knowledge, as put together by a prepper blog
500 Free Printable Workbooks and Self Help Guides, as put together by a mental health resource
Github's Awesome section - aggregates on resources for generally tech oriented topics. Primers on coding languages, getting better at any kind of programming or computer oriented task, and a few things related to interviewing can be found here!

Here's a few awesome lists I'd recommend from Github as well:

Awesome Privacy - privacy focused alternatives to everyday apps like fitness trackers, email, and search engines
Awesome Stock Resources - sites for stock photography and video
Awesome Self-Hosted - resources for self-hosting applications on a home server, including maintenance and storage.

Do you have a masterpost that's been helpful for aggregating information? I'd love to see it! ^^
Entry tags:

Introduction and goals


Name/Pseudonym: flori or florianschild


Hobbies/interests: Some hobbies include The Sims, reading, vegan cooking, sewing, learning Italian, fitness, painting and papercrafts. Interests... hmm it might be easier to list things I'm NOT interested in. I'm curious to a fault.


Do you data hoard? If so, what do you hoard?I have a small collection right now that includes ebooks, youtube videos (mainly fan made audiobooks that I've enjoyed), sewing patterns, and some random PDFs and images that aren't well-organized.


Why do you data hoard Mainly fear lol. But seriously, my interest in data hoarding is because over time I've seen things I love disappear from the Internet forever and it makes me sad. I used to naively believe that once something was on the internet it was forever, but that's not true! At least not without data hoarders out there obsessively screenshotting and saving everything.


Anything else? I'm very interested in how people are storing and organizing very large collections that you don't need to access often (for example, I'm hoping to download and store a MASSIVE collection of youtube videos, likely 300-400 hours worth of video, as a personal backup in case a particular channel ever went offline. I'm exploring the best way to store that, considering I'll hopefully never actually need it. If you have any suggestions, I'd appreciate that!)


Another of my goals for data hoarding, at least for now, is around preserving aspects of my very tiny fandoms. I've noticed that things I bookmarked just few years ago are already gone with no trace and sometimes I don't even remember what exactly it is that's missing.

Finally, I have a goal of preserving my own internet activity more consciously. I like the idea of screenshotting everything I write and all the ephemera of my digital activity. Comments on random articles and videos, posts on Reddit, reviews of products... I'd love to have ALL of that stored in one place just for personal satisfaction.
volkameria: Rodney (SGA) explaining something to someone offscreen (pic#rodney_explain)
[personal profile] volkameria2025-01-20 09:41 am
Entry tags:

Fanfic Downloading Tools

Hello fellow hoarders! Wanted to quickly share 2 tools I've been using to archive fanfic from around the internet.

1. Thanks to bites' recommendation, I've fallen in love with FanFicFare. FFF is an addon for Calibre, which I mentioned in my e-book/audiobook archival post. It's easy to set up and use and works for nearly every fanfic site out there! I say nearly every site because there are a few it will not download from (or at least easily), which are documented on their Github page. Included in that list is fanfiction.net; while it is possible, it relies on browser cache shenanigans that don't always work.
2. Fichub is a lightweight tool that's especially helpful for sites that don't play well with FFF - the biggest of these being fanfiction.net. It also supports AO3 and a few other smaller sites that FFF does not have full support for, and if you don't want to install Calibre, this is an easy way to get EPUBs. If you're not an EPUB person, Fichub will also let you download stories as pdfs or mobis. Additionally, Fichub is open source (link goes to their Github repo). Between these two, most general purpose and some specific fandom sites should be covered.

biteshelter: Drawing of a white cat with a bow tie (Default)
Entry tags:

Intro

Name/Pseudonym: bites

Hobbies/interests: Programming, drawing, Linux, music

Do you data hoard? If so, what do you hoard?

I wouldn't say I'm much of a data hoarder as of yet. I'm used to downloading stuff but not keeping it long-term. With that being said, I do have some collections that could be considered hoards.

Hydrus Network keeps my collection of art by other people. I keep Calibre libraries for things like fanfiction, books, guides, research papers, zines, and keepsake games. There's also Flash games (I used to use the developer version of Flash player to load them, but now I use Ruffle), music (organized with Beets), sound samples, some random useful code snippets, and free graphics I've used. All of my own writings, art, etc. are also hoarded.

Why do you data hoard?

I started off collecting files for offline access. When I didn't think I needed something anymore, I'd delete it. After experiences of trying to find stuff I remembered seeing and then failing to find it or finding out it had been taken down, I started keeping what I considered the "most significant" files I got. Learning about data hoarding (and censorship, and websites disappearing) made me realize the importance of keeping data in general.

Hoarding my own creations, which I've always kept long-term, is mostly for preserving my improvements and work process over time. A large part of hoarding for me is keeping multiple versions of things. Sometimes that's done by just making copies while working, but nowadays I use git for a lot of my writing and coding.

Anything else? Excited to be here!

Here are a few extra things I was reminded of while writing this: Flashpoint Archive is a downloadable archive of Flash media with its own player, MusicBrainz Picard is another music tagger/organizer I've seen recommended for being simpler than Beets, and this is an open access research paper on version control for artists.

volkameria: Rodney (SGA) doing computer stuff (pic#rodney_computertime)
[personal profile] volkameria2024-12-26 07:39 am
Entry tags:

Tools for Podcasts, Audiobooks, and EPubs

This is a tutorial and recommended tool list for managing podcasts, audiobooks, and Ebooks. I hope you all get some use out of these! If you have any additional questions, comments, or recommendations, please let me know.

Read more... )
volkameria: Rodney (SGA) introducing Elizabeth to a great meme he found (pic#rodneyelizabeth_computers)
[personal profile] volkameria2024-12-25 08:09 pm
Entry tags:

Public Introduction

Name/Pseudonym: Volkameria/Vol

Hobbies/interests: Video games, writing, baking, homemaking, reading, TTRPGs... basically if it's domestic and/or nerdy I'm on it.

Do you data hoard? If so, what do you hoard? Media of all kinds for personal use! TV shows, movies, ebooks, podcasts, tools, fanart/fanfics...  if I like it, I probably have a local copy. I also like to hoard history that has been censored in the past, and history that could possibly get censored again; things like 1984, histories of terrorist organizations, Project 2025 and Christian Nationalism, etc.

Why do you data hoard: This is a combo of a few things. I firmly believe in having access to high quality and factual information. With the combination of rising authoritarianism, the loss of past generational resources, and the degradation of search engines due to AI slop, I still want to have access to information, guides, entertainment. With everything being a "license" to view something rather than just owning it as well, with content removed even if you pay for the service, there's simply no guarantee something you like can be available - which is just weird! 

Anything else?
I tend to be vaguely paranoid, so my apologies if I'm cagey around certain things! While I'm not very good at servers, I'm comfortable with scripts (especially Python), command line interfaces, and debugging code problems, so I'm happy to help with anything on that front.
bedes: Icon of Kangel from Needy Streamer Overload whistling (kangel)
[personal profile] bedes2024-12-22 03:47 pm

Idiot-Proof Methods of Datahoarding

A lot of datahoarding resources assume that you're intimately familiar with command prompts, webcrawlers, scripts, and more. But what if... you're stupid? What if you're extremely unfamiliar with those sorts of things, looking to have your hand held through the entire process? What if your name is Azure and you have an account called bedes on Dreamwidth.org? (Wait, that's getting too specific.)

Well, these resources are for you! Idiot-proof resources I've gathered for archiving stuff online! As tested and approved by an actual idiot!

(Quick disclaimer: I call myself "stupid" and "an idiot" lightheartedly, and as a morally neutral descriptor. Being stupid about certain things isn't bad! Which is why I'm making this post!)

Resources for Dummies! (ordered loosely from most to least intuitive) )

If you know of any data-hoarding / archival resources that wasn't mentioned here, and you think even a total Python-illiterate doofus could get working, link it in the comments below! (Also, please include if it involves or requires any downloads, just because I think that's useful info.)
doranwen: female nerds, rare and precious (Default)
[personal profile] doranwen2024-12-20 12:42 am
Entry tags:

Intro


Name/Pseudonym: Doranwen

Hobbies/interests: fandom (over 20 of them), Linux, education

Do you data hoard? If so, what do you hoard? I hoard fanfiction (I adore ao3downloader and fichub-cli), movies (generally TV films that didn't make big budget), my favorite TV shows, ebooks on a variety of subjects, a few odd programs and games, and I have 14 terabytes of Yahoo Groups data on my hard drives because I got sucked into saving it (see [community profile] yahoogroups if you're curious about that project).

Why do you data hoard? Because you can't trust data to be available when you want it, sometimes because no one will sell it to you when you want it, but mainly because no one else saves it and then when you want it it's lost forever. Plus it's wonderful to have tons of stuff at your fingertips when the 'net is down; you might miss talking to someone but you have heaps you can watch or do.

Anything else? If you're interested in the Yahoo Groups project, it can use a ton of help tagging the groups so they can be organized properly (and uploaded to the Internet Archive). Not super hard, just takes the right kind of brain to love it.

Random Question #1

 What's one piece of advice you'd give to people interested in hoarding data?
malymin: An image of Miho from Season Zero of Yu-Gi-Oh with hearts around her. (Miho)
[personal profile] malymin2024-12-16 09:04 pm
Entry tags:

Intro


Name/Pseudonym: Schaf, Malymin
Hobbies/interests:
Do you data hoard? If so, what do you hoard? I don't think I have as much of a "hoard" as I'd like, but I do tend to squirrel away PDFs (ranging from articles to entire books) and random image files on flash drives and hard drives. I also like to keep anime episodes (for easy watching without internet access) and abandonware games on my PC.
Why do you data hoard? When I was a kid I noticed that websites I loved would often go down and never come back, because the webmaster could no longer afford to keep the domain. As a result, the internet not being a place of permanence struck me pretty early on. In addition, two formative pieces of media for me (the 1990's "Petz" games, and the 1998 Toei adaptation of "Yu-Gi-Oh!") only manage to continue to exist and be experienced due to piracy-as-preservation. If something goes out of print in an official capacity, it's gotta be preserved and backed up somewhere. Also, I like researching various topics, and it's cheaper to build a little nest of free PDFs than to splurge on books, as much as I prefer reading paper to reading a screen...
Anything else? I'm probably not as cool or good at this as most people here. u_u; Hope you'll have me anyways!

luckyzukky: a pink color sony mini disc (stock | md #1)
[personal profile] luckyzukky2024-12-15 02:51 pm
Entry tags:

how to use MakeMKV to losslessly rip your DVDs and blu-ray discs!

hi hello! are you interested in ripping a copy-protected DVD or blu-ray, in the best quality possible, without having to jump through hoops to break said copy protection? i'd like to introduce you to MakeMKV! it's a nifty little multi-platform program that's capable of cracking most methods of disc copyright protection, and rips the entirety of a disc (or just certain chapters, if you'd like) into lossless MKV format.

this guide serves as an introduction to using makeMKV, as well as re-encoding the MKV files it produces into a codec friendlier for media streaming, local or server-based (for example, if you use VLC media player, or use a media server such as plex or jellyfin), and to shrink the (somewhat large) resulting file sizes with little to no visual loss in quality using ffmpeg.

click here for guide )
gze: (default)
[personal profile] gze2024-12-15 06:21 pm
Entry tags:

Intro! Data Hoarders Unite!

Name: We're a plural system; our main hoarders go by G (they/them) and E (xey/xem/xeyr) respectively!

Hobbies/interests: alterhumanity, video games, writing, canines, D&D, toku, overthinking, drawing

Do you data hoard? If so, what do you hoard? Yes! We consider it our dragon's hoard. In no particular order, we've saved fanart, videos, music, fanfic, photos, personal records, health tips, community essays, screenshots of cool or informative posts we've come across, D&D notes, references, manuals, Game FAQs, language study guides, memes..... One of our oldest files is from 2003 that we found on an old floppy disk, a terrible school PowerPoint filled with pictures of George Bush, clipart, and Hamtaro characters????

(Organizing those files into folders is a very calming (and sometimes time sinking) activity, would recommend when needing an anxiety distractor.)

Why do you data hoard? For personal historical preservation, mainly! For some things it was to be able to watch offline in case the internet failed. Once we started really understanding that stuff online is not permanent and can be taken down at any time, and that hard drives can fail (or be stolen, both of which have happened to us before), it really made us paranoid want to archive stuff we really enjoy, or even stuff we don't fully enjoy but still want to look back on. This also includes our old chat histories, emails, screenshots of sites we were on in the past that we want to keep record of, the list goes on. Our first memory of doing this is keeping notebooks we passed back and forth with a friend in elementary school, and we are super grateful to still have them!

We also agree it's important to 'circulate the tapes' of media that is otherwise inaccessible, because capitalism sucks and preservation is very important. With unlimited time/resources/energy we would love to scan/rip all our rare media and archive those too.

Anything else? We're taking this love to grad school and starting a Master's in Information Science this coming spring! We hope to learn even more about archival and preservation that we can then take to our hoarding. For now, we're excited to learn new techniques and see posts from others who understand our niche interest! Already it's fun reading the intro posts to see what others have been hoarding!
Entry tags:

Man I love organizing data

Rex Name/Pseudonym: An ongoing debate but Monsters, Anomaly, and Pomeg work

Hobbies/interests: Pokemon, alterhumanity and plurality, web development, art and writing, video games, cartoons

Do you data hoard? If so, what do you hoard?: We're still getting into it and might not consider ourselves a data hoarder per say but we've been adjacent to data hoarding for some time now. We discovered Hydrus Client and Zotero a few years back so we've been steadily building a collection of saved images, websites, and music.

We have a external HD on the backburner of stuff to buy, but for now we have two 1TB SSDs with the second being regulated to the media SSD. Though we've been learning how to use a web scraper which is going to help with the process of getting and tagging our files at least a lot easier.

Why do you data hoard: It started with the annoyance of not being able to find what we were looking for again once we lost it - both with websites and images. We aren't too as focused on preservation so much as the freedom of having our own library, but being on the internet for long enough means we've lost a lot of things we once liked to being deleted or privated.

Anything else?: We're a professional web developer but it's more "thing we do to make money" than thing we're good at. Some day we will learn how to do things with Python to make both of these easier.

We're also a multiple system, hence the plural pronouns.

luckyzukky: oguri yui from akb48 (akb | yuiyui #1)
[personal profile] luckyzukky2024-12-14 09:17 pm
Entry tags:

server freak intro

name: melo

hobbies/interests: self hosting and other related nerd stuff, k-pop & j-pop idols, tokusatsu, dc comics, and more

do you data hoard? if so, what do you hoard?: i hoard way too much! i run multiple media servers, some SFTP servers, and have a ton of stuff spread across different hard drives. it's just about anything - shows/movies/music, idol media, comics/manga, books, software, and so much more.

why do you data hoard: i strongly believe in preservation, and especially that the proliferation of data is key to preservation; more than physical, more than one copy on archive.org, proliferating data and sharing it with others, multiple people having the same copies of the same thing, is the ultimate way to preserve something. and if you couldn't tell by that, i'm a big fan of torrents lol

anything else?: ask me about servers :)
bedes: An icon of Kabru from the Dungeon Meshi manga, smiling bashfully (kabru)
[personal profile] bedes2024-12-14 07:43 pm
Entry tags:

Storage is cheaper than regret

Decorative divider


Name/Pseudonym: Azure or Bede!

Hobbies/interests: I write, draw, code, and edit (gifs, images and videos), among other things. I really love animation and video games!

Do you data hoard? If so, what do you hoard?: I do! I hoard my own stuff (the previously-mentioned drawings, writings, coding projects and edits, as well as regularly backing up my and my husband's social media accounts), plus stuff related to my fandoms! I either archive things publicly (via one of my personal sites or archive.org), or privately (on Iris, my 1TB external drive).

Why do you data hoard?: I really got into data hoarding after the whole fiasco with Warner Bros removing several TV shows from their services for tax writeoffs. It was the catalyst to me realizing that I can't trust companies to preserve what I love; they'll make entire pieces of art illegal to watch, just to save a few pennies.

Anything else?: I'm an extreme newbie when it comes to datahoarding (as you can probably tell by me only having only one external drive), and I'm doing it on a very tight budget (which is why I only have the one external drive, lol). I hope I can still say some things of use when it comes to this topic, though! I'd also like to mention that I'm mentally disabled, so I may communicate in a strange way, but I promise I'm never trying to be rude or combative or passive-aggressive or robotic or etc.
 

Decorative divider


Entry tags:

Intro Post

Name/Pseudonym: Bernie / Sci

Hobbies/interests: drawing, art in general, film, history, science fiction

Do you data hoard? If so, what do you hoard? Films, TV shows and books, besides my own stuff.

Why do you data hoard Preservation. Nobody owns anything anymore, and companies can pull the plug, and your favourite TV show is gone. Why yes, I am into the Lost Media (TM).

Anything else? I had all of Jacqueline Hill's available filmography (she was in Doctor Who), until I did NOT back up my videos and lost an episode of Angels she was in. I can't find it anymore.
Don't be like me. Do frequent backups.