Fanfic Downloading Tools

[volkameria]

Hello fellow hoarders! Wanted to quickly share 2 tools I've been using to archive fanfic from around the internet.

1. Thanks to bites' recommendation, I've fallen in love with FanFicFare. FFF is an addon for Calibre, which I mentioned in my e-book/audiobook archival post. It's easy to set up and use and works for nearly every fanfic site out there! I say nearly every site because there are a few it will not download from (or at least easily), which are documented on their Github page. Included in that list is fanfiction.net; while it is possible, it relies on browser cache shenanigans that don't always work.
2. Fichub is a lightweight tool that's especially helpful for sites that don't play well with FFF - the biggest of these being fanfiction.net. It also supports AO3 and a few other smaller sites that FFF does not have full support for, and if you don't want to install Calibre, this is an easy way to get EPUBs. If you're not an EPUB person, Fichub will also let you download stories as pdfs or mobis. Additionally, Fichub is open source (link goes to their Github repo). Between these two, most general purpose and some specific fandom sites should be covered.

Comments

[toothpastepancake]

Oooh! Thanks for this!

[doranwen]

I definitely recommend the use of FicHub! It has a script-based tool - fichub-cli - which is very handy for running large batches of links.

If someone's trying to grab fic from Ff.n, for instance, the solution I've found is to use Firefox with two extensions: Fanfiction Tools, and Link Gopher. I'll run through a quick guide below for anyone who wants to do this en masse.

Set your filters on ff.n for the fandom that you're trying to grab (I haven't tested to see if this works with a search), and turn on Fanfiction Tools (I leave it off most of the time because I find some of its features a bit disconcerting if I'm browsing, lol). Then it will automatically load the next page every time you hit the bottom. (If you have a LOT of pages, like if you're trying to grab the HP fandom or something ridiculously large like that, you may want to get a small magnet or something you can set on the Page Down key so it'll keep hitting the button for you while you go do other things.)

Once you have the entire list of links you want, then you click on Link Gopher, choose Extract Links by Filter, and type in "/1/" (minus the quote marks) into that. That will filter out all extraneous links except for one each at the top and the bottom of the list. Those are easy to remove manually later. The rest of the links are all the story links that you're wanting.

Copy/paste the entire list into a text file, remove the extraneous links, and you've got the list ready to feed into fichub-cli! I fed it all of HP for a friend once (which took three months to download, lol), so I know it can handle really long lists. (You just don't want to feed it a list longer than your computer can download in one shot before it has to restart.)

[atamascolily]

FanFicFare is incredible, I love it so much.

I've been using a Webpage to EPub extension for FFN, but Fichub looks even easier to use, so thanks for the rec!