How to organize your JAV collection?

[paparoach408]

Coming back to this thread. I have been working slowly but surely over the past few days to get more JAV. The fact that offkab is uploading plenty of 10GB+ file sizes certainly helps. I have a 1TB Seedbox I use for this stuff, and have only recently added a new 500 gigs to it. I go through phases, and it seems like my JAV phase is beginning again. Typically lasts anywhere from a week to 3 months at a time. Heres hoping I am able to boost myself over the 20TB mark this time around (At around 18TB as of now). My download speed has gotten better, and going 24/7 with no interruptions, I can manage over 400 gigs a day (close to 500 actually). So, anything interesting going on in the JAV world?

that's insane, and amazing!

 

[LoliKing25]

Going by typical 2G / hr for FHD, you are downloading 200-250 hrs of porn every 24 hours.

Haha. Not quite, as I am also doing TV, Movies, Cartoons, Etc. But, I am close to 600 gigs waiting to be downloaded from my seedbox (none going yet) (All JAV).

 

[summerss]

The way I do it for my collection is by using these 5 essential softwares


1)Ultra Search

One of the fastest file searching software with wildcard capabilities, I put all my finished downloaded files at a directory and then use UltraSearch to delete all misc "advertisement" files that usually come with the download by searching just for the extension (example .mht or .url). I usually add 50+ downloads in my queue and can't be bothered to select just the movie files and photos

2) Bulk Rename Utility

After clearing all the junk files my next step is to name them properly and removing certain texts like [thz.la] or p2p.wowweeelubba.SNIS-404 for 200 folders isn't fun. This is one of the best tool for the job.

One of the most used function by me is to Rename files from foldername automatically, so I only need to rename each movie's folder properly. Also I use a batch file to move all movies that came without a folder into it's own folder according to the files name

3)JavMovieScraper by DoctorD1501

One of THE best data scraper I've used to date, though the creator has gone MIA for at least a year but someone by the name of Wizell has taken over the project.

This java program is able to scrape 99% of JAV movies from sources such as DMM, Javlibrary, Javbus, and also uncensored ones from AVentertainment, Caribbean, Heyzo, 1pondo.

What it does is download Covers and creates a .nfo file with multiple Actress names, english titles, subgenre, year of release so that you can easily search for whichever actress you want a list of, then again the accuracy of this data is strongly depended on the sites it scrapes for, personally I prefer scraping from Javlibrary as the actresses list are mostly accurate and I can always submit a change to that site if I find the data to be inaccurate.

Also to use this tool you must make sure the files are named *CORRECTLY* or you'll get wrong titles scraped

4)DupeGuru

I use this to find any duplicate movies I may have downloaded

5)KODI

Finally what's the point of all that data scraping if you don't have a way to properly 'view' them? Previously known as XBMC, KODI feeds on all the data you've scraped and you can browse all your collection by Year, Actresses, Genre and view all the movies via covers. Very customizable experience

I did this but I used plex instead. Kodi confuses me.
Everything looks nice and clean now. If you know regex it makes sorting even easier. If only western content was named the same because its almost impossible to sort.

 

[LoliKing25]

I currently have a backlog going bck to May of movies to download. It is currently at 10TB+, and I am able to get up to 10TB per month, but I mostly get around 8TB a month. The majority of which is not JAV, but TV and Movies. I have 2 months of backup of not even downloading the torrent files,but a select few. We shall we going forward, as I want to be done with the backlog up to last month by end of year. We will just have to wait and see.

 

[javconnoisseur]

I did this but I used plex instead. Kodi confuses me.
Everything looks nice and clean now. If you know regex it makes sorting even easier. If only western content was named the same because its almost impossible to sort.

I know a little bit of regex but have stopped using it ever since I made a small error on the code, wiped out some high def content instead of the SD content

However my scraping have stopped for months now after Javlib started using cloudflare, development of the javscraper has stopped (again!) and I don't really want to scrape from another source due to the different naming methods of the actors and lack of for HUNTER series

 

[bigloser0]

I know a little bit of regex but have stopped using it ever since I made a small error on the code, wiped out some high def content instead of the SD content

However my scraping have stopped for months now after Javlib started using cloudflare, development of the javscraper has stopped (again!) and I don't really want to scrape from another source due to the different naming methods of the actors and lack of for HUNTER series

Used to watch HUNTER series from years ago and they were indeed good.

 

[ding73ding]

Regex is indeed difficult but you just have to learn test/debug your code carefully before any mission critical operations, even if you avoid tricky tools like regex.

I now use javbus.com and pornav.co as backup. In fact, after many months of stable usage (no code development) just yesterday I added as pornav.co as automatic fall-back (previously it was manual fall-back). For some reason javbus.com lacks any data of some tens of vids that were just released, even top series such as STAR-. So now my scrapper first go to javbus and if not found goes to pornav.co. The downside of pornav.co (the reason it's fall-back) is that often it mistakenly include text like "Limited release" or "Bonus attached" to the vid's title. The final fall-back is DMM itself. Some old vids are just absent from most other websites and if I collect such a vid I may have to find it on DMM, then Control-A, Control-C to copy the page's text and paste into my scrapper which will pick off the data points to fill into my database.

I'm afraid you shouldn't be in the scrapping game if you (over) rely on English. I understand most on here would want English on the UI, but underneath for the data you should use Japanese as gold standard reference. I think even most public website, even your preferred choice javlib, aren't internally consistent with English. For my own mini-project (actually a module) I keep a database of AV idols using Japanese as gold reference but has the option of displaying all English using a database. To fill the database I use minnano-av.com for English naming method. So far I fill it manually but one day (someone) can write a scrapper for it. (average of a couple names per week, not really worth the coding effort)

Lack of (idol data?) for HUNTER? Easy, sougouwiki.com for fairly (not totally) complete actors in HUNTER, AP and other fake amateur series. So far my habit is whenever I collect a (HUNTER etc) vid I manually copy the idol info from sougouwiki into my scrapper. So for vids I don't collect, my database may be incomplete... oh well...

If my scrapper + database had more users/value/demand, it's worthwhile to write scrapper for minnano for idol data, and sougouwiki for HUNTER/AP vids

 

[Casshern2]

I'm very glad to have found this thread (and all of you) just as I started to enjoy and collect.
I've been using the JAVMovieScraper that was recommended and conveniently located on GitHub.
I'm so glad I found this thread while Googling ideas.
I mainly use this as a means for the organization of my favorite digital titles for use on Plex/Emby.
It works out well on both using the movie.nfo, poster.jpg, and fanart.jpg structure.

However, I find myself overwhelmed with OCD and end up using a combination of a few things,
I start with the JAVMovieScraper but for titles that are not giving me exactly what I want,
then I use tinyMediaManager to make bulk & finishing touches.
It utilizes the data pulled in from r18.com via scraper but not everything is perfect.
Abe Mikako is definitely more hardcore than an NC-17 rating.

View attachment 1562936

It's a great little program that I donated to, but are against it's use organizing porn.
What? I know, right?
But it works so well connecting my watched Anime, TV Shows, and Movies to Trakt.tv
They all work well together, and I even integrated Taiga for my anime download workflow.
But who cares about all that right?
We've got porn to organize.

Here's where it gets weird. I use an entirely different application for my physical discs.
I fell in love with CLZ's application that let me edit and play around with the CSS interface.
Yes - I was a dirty pirate hooker and "tested" out the program for free.
It also works well together with the previous apps all using the same .nfo's and pics.
I also make stuff, and design things as a career.
I know when to open my wallet for something well-made.

View attachment 1562937

I only have 24 physical titles.
I can only imagine what it would feel like to already have a buttload of digital titles collected.
Is that my future? lol I should buy some more storage nao.

As for now, I am 100% filled with happiness browsing my temple of Japanese idols.

Glad to have you here! Always good to have others join in with new ideas!

 

[Muz1234]

Can you put a video of how you organize the JAV collection in computer? Maybe a tutorial video, of install the software to organize the collection? I am confuse, as how can a JAV database be installed in your computer as you have downloaded a JAV film, for example?

 

[Casshern2]

Can you put a video of how you organize the JAV collection in computer? Maybe a tutorial video, of install the software to organize the collection? I am confuse, as how can a JAV database be installed in your computer as you have downloaded a JAV film, for example?

http://www.lifewire.com/databases-for-beginners-1019643

 

[Muz1234]

How to organize it by the use of SQL? Does SQL has anything to do with the program libraries above you posted?

 

[Casshern2]

Really, SQL (Structured Query Language) in its simplest form can be used to query any database that supports it. You don't require SQL Server to use the language, only to leverage enterprise level databases. For example, I use it in my little project (VERY little) that I still get to every now and then. At least I still feed it data but haven't done anything other than that recently.

https://www.akiba-online.com/thread...lication-work-in-progresspretty-much.1784224/

I'm using SQL to query against a simple CSV file. The reason was for others to use it without having to have Office for the sake of Access or Excel. I first, years ago, was working on an ASP Classic site run locally off of an Access database, but I later halted on that.

 

[javr]

I use JAVMovieScraper by DoctorD1501 to create XBMC nfo data. I use Plex with XBMCnfoImporter plugin to add media to libraries. Together its pretty sweet and i can sort by actor/genre/studio etc

Does this support scraping of kanji info?

I use Kodi + manual .nfo mainly for presentation + semi-organised.
Work best if you're doing HTPC.
(Different profiles for JAV and gravure idol libraries)

 

[LoliKing25]

Does this support scraping of kanji info?

I use Kodi + manual .nfo mainly for presentation + semi-organised.
Work best if you're doing HTPC.
(Different profiles for JAV and gravure idol libraries)
View attachment 1573048 View attachment 1573049 View attachment 1573050 View attachment 1573051

I have heard of Kodi before. How do you recommend for someone who refuses to do a NAS, and just use 23 External drives?

 

[javr]

I have heard of Kodi before. How do you recommend for someone who refuses to do a NAS, and just use 23 External drives?

Logically, you can still add your videos into the libraries. It will still display as it is, just that you are unable to play the videos when it's unplugged.
Also, do note that your paths to the videos have to be consistent.(drive:\\directory...\name)
Drive letters assigned to the USB externals can change depending on availability and sequence.

 

[LoliKing25]

Logically, you can still add your videos into the libraries. It will still display as it is, just that you are unable to play the videos when it's unplugged.
Also, do note that your paths to the videos have to be consistent.(drive:\\directory...\name)
Drive letters assigned to the USB externals can change depending on availability and sequence.

What Kodi Skin is that? Is there a special one for JAV?

 

[javr]

What Kodi Skin is that? Is there a special one for JAV?

It is the confluence skin. I just change to custom background.

 

[IdeNali]

I recently bought myself a NAS (RAID 5 4x8TB) and wanted to start using Emby (as the media server) and Kodi for browsing/playing my collection.

I already fiddled a little with Kodi to test it out and now I'm wondering how you guys search for JAV in your collection in Kodi. My hopes were that I could do a Tag based search but that only works with filters and not with the global search. This seems very inconvenient because currently I'm browsing my collection through tags. With my tagging system I could easily find what I want without knowing the title or the id in seconds. Is that even possible with Kodi?

Any advice is appreciated!

 

[Casshern2]

Wow, there are a few places to discuss something like what I'm working on now, but, I think it really belongs here since I'm using this to keep track of how I'm organizing my JAV collection? Yet, I've talked about this very subject, or at least mentioned what I use for it, in a number of places I'm afraid, simply because it felt then the place to do it...

JAV Database HTA Application: Work In Progress…pretty much
https://www.akiba-online.com/thread...lication-work-in-progresspretty-much.1784224/

JAV Database
https://www.akiba-online.com/threads/jav-database.399610/

Completionism
https://www.akiba-online.com/threads/completionism.1787353/

Do you burn JAV files into DVD
https://www.akiba-online.com/threads/do-you-burn-jav-files-into-dvd.1797059/

Hopefully no more than that, but, I got tired of searching. The reason for today's post to bring back this thread? Progress! Or, at least, a bit of it. I'm currently working on a new PHP version of the HTA app I was working on (that lingered until I stopped working on it!). This new iteration is pretty bare bones visually, I'm working on the mechanics first this time, I can pretty it up later. I found a nice way to learn PHP using a small portable web server called USBWebserver (https://www.usbwebserver.net/webserver/). It runs Apache, MySQL, PHP and PHPMyAdmin all from a local drive/folder or off a USB, so, your site can be portable. Of course this can't be used for anything commercial as it can't handle it, but, for what I'm using it for it is pretty zippy!

I've always wanted to learn PHP but was too lazy to set things up. This thing was a godsend! Below is a link to a quick < 1min video showing me navigate around quicker than I should have. Below that is the play-by-play for any that need it. I'm so far just working out how the navigation will work, still need to make the keyword search a bit more robust, just finished re-coding for the links on the results pages to work correctly. For now the keyword search searches against DVD Code, Star, and Disc fields together, so say if I searched for 'mot' it will bring back all the MOT-XXX titles and any other DVD Code that has mot in the name, plus any JAV Star that has mot in her name. Technically, if I had any storage discs with a label name that has mot in it, that would come back, too. All of these are from from fields in the database, currently manually updated via scripts that get just data from the files, so, no web scraping at play here quite yet, either something I've done or any other available method.

https://mega.nz/#!CJtwXIBR!2dg057WTEh709ScwB-uL21kUbfxk-1Q-B0TRGNOZ3DY

Here is what you see me doing if you grabbed the video. This is VERY basic PHP, mind you.

Load site
Clicked GVG DVD Code/Prefix link on the main summary page to load all GVG-XXX titles
Clicked to second page of GVG titles
Clicked back to first page of GVG titles
Clicked on BD073 link to show all titles on (blu-ray 50GB dual layer) Disc BD073
Clicked TASKS DVD Code/Prefix link to load all TASKS-XXX titles
Clicked on BD052 to show all titles on Disc BD052
Clicked OAE to load all OAE-XXX titles
Clicked on BD080 to show all titles no Disc BD080
Clicked TUNA to load all TUNA-XXX titles
Entered Keyword 'jufd' to load all JUFD-XXX titles
Clicked to second page of JUFD titles
Entered Keyword 'ryoko' to load all titles with Ryoko Murakami
Clicked SPRD to load all SPRD-XXXX titles
Entered Keyword 'BD018' to load all titles on disc BD018
Entered Keyword 'exit' to load blank site

Entering 'exit' is how just how I used to close out the HTA app, this really just goes blank because nothing in my database has exit in the value. Once I get the mechanics working a bit better I can start making it look more presentable. Then once it's ready for anything further and I'm allowed by the mods I can start a thread for it like I did the HTA (and hope it doesn't get abandoned like that one...ahem... ). Some can maybe use it, change it, or whatever, for their needs.

edit: probably appropriate to provide some screenshots.

 

[Casshern2]

I cleaned up slightly the data under each title, removing the label from a separate line and made just that part of the DVD code the hyperlink. I also have it expanding the div when the browser is resized.