JAV database

[CodeGeek]

Good to see how many people are active now. When I started this thread I thought I'm the only one - some kind of lone wolf.
Unfortunately I didn't have much time since Christmas because of work and private life. Seems I will get some more time from now one at least the next 4 or 8 weeks.
Instead of discussing my ideas here I guess it is better to create something first I can show to you all. I will post here again if I have something.

 

[cyberzen]

I've got

1) Sync working tentatively (alpha)
2) Screenshot generation
3) detail view

you can download the windows version here http://tagu.in/windows/Tagu.zip. Don't expect much for the syncing at the moment, since there's not many people using the app, other users would not have uploaded much data. I'll be adding my own data soon and also a website for users to browse the content online.

 

[Casshern2]

...Anyway the key point is that I have no idea which site could serve as a good source for scraping and/or crawling. DMM has a fatal issue, AFAIK, the product code listed on their product pages is mangled from the official product code. I don't see a simple algorithm to extract the official product code from the mangled DMM-specific code.

Simple, probably not, but by the URL there is some parsing you can do:

"http://www.dmm.co.jp/digital/videoa/-/detail/=/cid=118sga00005/"
"http://www.dmm.co.jp/digital/videoa/-/detail/=/cid=118rdt00181/"
"http://www.dmm.co.jp/digital/videoa/-/detail/=/cid=h_108mobao00030/"

Below is my quick and crude way to get the DVD code from the URL. Maybe using regular expressions it can be done more efficiently but I'm fried from work. Maybe you can frankenstein it into something useful. If not, it was kinda fun seeing if it worked. There would of course be some that this alone won't parse correctly. Don't laugh at me too loud. Just seeing if I could do it.

Dim intPos As Integer
Dim strDVDCode As String
Dim strCode As String
Dim varChar As Variant
Dim i As Integer

strDVDCode = "http://www.dmm.co.jp/digital/videoa/-/detail/=/cid=13gg00276/"

' // find the string "cid=" because after that is the dvd code
intPos = InStr(strDVDCode, "cid=")

' // just grab everything after the position of our found string
strDVDCode = Replace(Mid(strDVDCode, intPos + 4), "/", "")

' // find the position of the underscore (if there is one)
intPos = InStr(strDVDCode, "_")

' // if there is...
If intPos > 0 Then
' // just grab everything after it
strDVDCode = Mid(strDVDCode, intPos + 1)
End If

' // look at each character until we reach a letter
varChar = Mid(strDVDCode, 1, 1)

' // we need to search each character until it is not numeric
Do Until Not IsNumeric(varChar)
' // add to our counter
i = i + 1
varChar = Mid(strDVDCode, i, 1)
Loop

' // if we counted instances of numeric characters
If i > 0 Then
' // grab everything after those digits
strDVDCode = Mid(strDVDCode, i)
End If

' // reset the counter
i = 0

' // now look at each charcter until we reach a numeric character
varChar = Mid(strDVDCode, 1, 1)

' // we need to search each character until it IS numeric
Do Until IsNumeric(varChar)
' // add to our counter
i = i + 1
varChar = Mid(strDVDCode, i, 1)
Loop

' // we should now have just the raw dvd code: however many letters in dvd code and 5 digits which include the dvd code at the end
' // so we show our code as parsed up to the beginning of the numeric digits _
Mid(strDVDCode, 1, i - 1) ' // the i from just above _
and the remaining 5 digits and we usually only need the last three, but for titles like VNDS...

If Mid(strDVDCode, i, 1) > 0 Then
strDVDCode = Mid(strDVDCode, 1, i - 1) & "-" & Mid(strDVDCode, i)
ElseIf Mid(strDVDCode, i + 1, 1) > 0 Then
strDVDCode = Mid(strDVDCode, 1, i - 1) & "-" & Mid(strDVDCode, i + 1)
Else
strDVDCode = Mid(strDVDCode, 1, i - 1) & "-" & Mid(strDVDCode, i + 2)
End If

MsgBox strDVDCode

End

 

[cyberzen]

As far as I know, censored DVD codes always have a 3 digit numeric code, it's the uncensored DVD codes that have more than 3 digits.

 

[Casshern2]

cyberzen, that is correct. What I mean is if you look at the URL for a title on DMM (using JUFD-329 as an example) you can see the recognizable label code of "jufd" followed by five digits.

http://www.dmm.co.jp/digital/videoa/-/detail/=/cid=jufd00329/

The last three complete the DVD code but it seems to almost always have five digits. I've seen some that have less or more. To get around that we can probably get the count of digits and handle accordingly, like I did for the VNDS titles in the sample I posted.

 

[cyberzen]

Ya I know, if you look at the cover image it will state the DVD code is JUFD-329, which follows the same format as torrent sites listing. I'm not too familiar with DMM or other japanese based DVD stores as I can't read japanese, but it's probably their own ID format.

 

[Casshern2]

ding73ding's comment prompted me to try my hand at it. Maybe he can figure something out for his purposes. Thanks, friend.

:hugethumb:

 

[cyberzen]

Just realized there are DVD code with 4 digits, DV-**** series and XV-**** series

 

[Casshern2]

Absolutely. This code handles that. ...it turns out

 

[elgringo14]

DMM is always inventing their own codes, don't rely on them.

 

[cyberzen]

Yea, I rarely browse DMM

 

[iori11]

I've got

1) Sync working tentatively (alpha)
2) Screenshot generation
3) detail view

you can download the windows version here http://tagu.in/windows/Tagu.zip. Don't expect much for the syncing at the moment, since there's not many people using the app, other users would not have uploaded much data. I'll be adding my own data soon and also a website for users to browse the content online.

new version new bugs on one pc (windows 7 "32")i got the error "use the correct version X86 X64" on another i got this after importing my movies

 

[cyberzen]

Are you really using a 32 bit cpu? or is it a virtual machine?

 

[CodeGeek]

Are you really using a 32 bit cpu? or is it a virtual machine?

I also have here a Windows 7 installation in 32 bit. Maybe it sounds a little bit unusual at first. But e.g. my netbook has a Intel Atom processor which has only 32 bit.
Of course my normal desktop PC has 64 bit.

 

[cyberzen]

I've updated the database with some JAV titles, your JAV videos need to be named exactly like the DVD code.

For example:
1) NHDTA-343.avi
2) SDMT-819.mp4
3) RBD-555.mkv

the extension does not matter, but the filename MUST match the DVD code, and yes you need the dash in the middle. I included a demo here https://www.youtube.com/watch?v=wWDHy5_1MPI

currently the syncing will not work for porn videos at the moment.

@CodeGeek, I was replying to @iori11. I can't even remember my last 32 bit CPU

@iori11, I will try to get a 32bit version compiled, but it is a lot of hassle to cross compile a C++ app from linux to windows if possible could you use a 64bit CPU.

 

[cyberzen]

I should also probably mention, the way the image directory is used in the options dialog.

There should be 5 sub directories in that folder, the app will create them automatically. Do NOT rename them or images will not show properly

1) actress
2) actress/small
3) covers
4) screens
5) thumbs

it's pretty self explanatory, I will be storing big pics of actress in the /actress directory, small profile pics of actress in the /actress/small directory, DVD covers will go in the covers directory, screenshots in screens, and small thumbnails in thumbs.

Currently I am hard coding the generated thumbs to be 400 X 225, after a successful import, you should have screenshots and thumbs generated in the respective directories. In the future a user will be able to configure thumb size and lots of other image generation options.

Also for people like CodeGeek who already has images for the DVD covers, you can just copy your existing images to the directories, they should follow the same title as your video exactly

for example:

your video is named SDMT-819.avi
your pic should be named SDMT-819.jpg

Currently, the cover, screenshot, thumbnail, compact and detail view should be working, the problem is my server will be overwhelmed if I include the cover images with the syncing. In the future I will include actress, actress profile pic and covers during the sync, so that after syncing you will have all the images available after a sync.

 

[CodeGeek]

@CodeGeek, I was replying to @iori11. I can't even remember my last 32 bit CPU

I know, I know. But you seemed to be amazed that some people still have 32 bit CPUs. But there are still a lot out of there it seems.

 

[Casshern2]

There are plenty of people out there with 32-bit machines. I'd wager there are still plenty out there with XP. Hopefully not as many since MS is dropping their security support of it. At any rate I think this a great group of projects going on.

 

[iori11]

I also have here a Windows 7 installation in 32 bit. Maybe it sounds a little bit unusual at first. But e.g. my netbook has a Intel Atom processor which has only 32 bit.
Of course my normal desktop PC has 64 bit.

Me its a old Pc who i use as a nas ( a 10 Tb jav NAS ^.^, what i am not addicted to jav at all, but now you know why i need a jav catalogue soft) if its a hassle I can install a 64 bit OS ^^ (the cpu is an Intel E7400 so it seems to support it) I rather have you working on this holy app than in os compatibility:th_059_:

you really did this on purpose so we switch to Ubuntu didn't you xD:shooock:

 

[cyberzen]

Eventually I will support all OS with 64 bit and 32 bit, but right now I can't even guarantee that this will be the final design. It took me about 3 days to figure out how to properly cross compile from linux to windows, which could have been spent working on features. Once I am happy with the features and the core logic I will start bug fixing and improving the OS support.