On the topic of digital games...

[Izret101]

I think you seeing separate folders/DBs as segregation/isolation/restriction/removal of functionality is the opposite of what would be happening.
Everything would finally get treated as it should be. Everything would have its own pages/search/etc tailored to it. Since hardware and then digital was added into a database made specifically with physical software in mind it has been like a square peg in a circle hole.

We have over 200 "consoles"(i use "" because digital delivery options are not consoles) listed in the database at this time because of the addition of digital delivery sources as a way to differentiate games from their retail counter parts.

Digital
1. As NES_Rules said we could cut a ton of wasted fields out of the fields on digital pages. Looking at a digital released game in the DB now it LOOKS like an empty page. Even when all applicable information is filled out.
1.a. I'm not a tech guy but i assume but cutting out all the waste fields(scans, data, boxes/manuals) would save resources. For example every single time someone does a missing image search it will hit a digital game page for 8 items when a digital platform won't have images for 4 of those fields. (Box front/back/ manual, game)

2. Having superfluous Box/Manual fields could be removed (quantity probably could too buy i know with Steam you can own multiple copies of the same game though i don't know why you wouldn't just want to trade/sell/give it away immediately.)
2.a. Again cutting the waste. We don't have to store empty values for an ever growing number of games in an ever growing number of collections. I assume this again would improve performance

3. Would increase ease of use. We still get Steam/GoG/Origin submissions for PC, PSN games submit for PSP/PS3/PS4, Games on Demand games submit for 360 with the variation title "Download" even though they do all have their own listing*
3.a. Saves time from staff having to reject a game and explain why. Also saves the frustration of members feeling like they wasted there time submitting this entry only to have it rejected.

4. I feel that in another years time the digital releases in the database will outnumber the physical releases. I also anticipate the creation of more and the support of more digital options. I really feel this would only further clutter search results and the system list and more than likely increase frustration of users.

Hardware
1. Much like the inclusion of digital "systems" clutters the list of actual systems the inclusion of Accessory, Cables, Controller, Memory/BackUp, System clutter the Genre field. When someone wants to search for a specific hardware item they shouldn't have to hunt through a list of completely unrelated things to find it.

2. Again ease of use. Items like cheat devices still get submit as Software. Wastes time of staff wastes time of user. No one feels good about having to reject a submission or getting a submission rejected.

3. Clutter/Ease of use. When you sort your collection (in real life) or go to buy from a store you don't look for the Playstation Move Controller between Pirates of the Caribbean and Power Gig.

Software (Physical) [Really doesn't relate at all but kind of needed to be here since i touched on the other 2... i guess.]
1. (PC specific)When digital started taking off and many people who heavily supported digital joined the site we ensured that they could view/track their collection in a way that worked for them by differentiating Steam/GoG/Origin/Etc from a PC retail release. Joyous cries were heard from the highest mountain to the lowest... low spot.

2. We added the virtual consoles/PSN/XBLA/GoD/XBLI for multiple reasons. People wanted to track that they owned I MAED A GAM3 W1TH Z0MB1ES!!!1 but didn't want it mixed in with their 360 games. While it is a game you play on the 360 Microsoft went through great lengths to let people know their system plays Retail/Games On Demand, Arcade and Indie games. Nintendo had done the same thing with the Virtual Console and WiiWare differentiating them in their own market places.

3. Variants have been(and probably always will be) a major point of contention. There are those completely against them. Those in favor of all of them and those in favor of some but not others. To a certain extent this problem has been dealt with for the users most irritated by variants returning when searching. When you want to know how many games were released for a specific system and learn that there are 8 versions of the same game bloating the number because of a potentially minor difference i can see how that would be frustrating.

*Initially all digital games fell under the umbrella of the system they worked on. Due to the ever growing demand that Steam and XBLA and PSN get their own support separate a split was made. That ended up opening up the doors for every other digital market place/distribution network for every other console.
When searching for an item to for any reason you wouldn't have to wade through a list of options completely unrelated to your search.

It is also worth noting we have never received complaints for not enough information being available in a search but do get complaints from too much making the user wade through to find the specific item(s) they want.



Really what it comes down to is a database split is in the best effort of:
Giving each item the right kind of attention
Keeping the database clean
Keeping everyone happy
Cataloging everything to the best of our ability. This is "The Classic and Modern Video Games Database" after all. Not the Cart-based only, CD-based only, Digital only, Hardware only, System/Company/Region specific Database.

[Shadow Kisuragi]

Wall of text - TLDR, as I'm jumping back on Street Fighter. The advocacy of a "digital database" is due to how we currently separate software and hardware, and drive the pages based on that. Separating Digital would just be an extension of that - that's all that's being asked. As for the collection tool, it would just hide anything that was disabled.

I've never been a proponent of separating even more, as it would just lend itself to performance problems when querying multiple databases as mentioned earlier. Instead, we should be extending the software pages to handle digital as well.

[Izret101]

If you can't be bothered to make a proper response why make one at all?

[Shadow Kisuragi]

I think you seeing separate folders/DBs as segregation/isolation/restriction/removal of functionality is the opposite of what would be happening.
Everything would finally get treated as it should be. Everything would have its own pages/search/etc tailored to it. Since hardware and then digital was added into a database made specifically with physical software in mind it has been like a square peg in a circle hole.

We have over 200 "consoles"(i use "" because digital delivery options are not consoles) listed in the database at this time because of the addition of digital delivery sources as a way to differentiate games from their retail counter parts.

Digital
1. As NES_Rules said we could cut a ton of wasted fields out of the fields on digital pages. Looking at a digital released game in the DB now it LOOKS like an empty page. Even when all applicable information is filled out.
1.a. I'm not a tech guy but i assume but cutting out all the waste fields(scans, data, boxes/manuals) would save resources. For example every single time someone does a missing image search it will hit a digital game page for 8 items when a digital platform won't have images for 4 of those fields. (Box front/back/ manual, game)

2. Having superfluous Box/Manual fields could be removed (quantity probably could too buy i know with Steam you can own multiple copies of the same game though i don't know why you wouldn't just want to trade/sell/give it away immediately.)
2.a. Again cutting the waste. We don't have to store empty values for an ever growing number of games in an ever growing number of collections. I assume this again would improve performance

3. Would increase ease of use. We still get Steam/GoG/Origin submissions for PC, PSN games submit for PSP/PS3/PS4, Games on Demand games submit for 360 with the variation title "Download" even though they do all have their own listing*
3.a. Saves time from staff having to reject a game and explain why. Also saves the frustration of members feeling like they wasted there time submitting this entry only to have it rejected.

4. I feel that in another years time the digital releases in the database will outnumber the physical releases. I also anticipate the creation of more and the support of more digital options. I really feel this would only further clutter search results and the system list and more than likely increase frustration of users.

Hardware
1. Much like the inclusion of digital "systems" clutters the list of actual systems the inclusion of Accessory, Cables, Controller, Memory/BackUp, System clutter the Genre field. When someone wants to search for a specific hardware item they shouldn't have to hunt through a list of completely unrelated things to find it.

2. Again ease of use. Items like cheat devices still get submit as Software. Wastes time of staff wastes time of user. No one feels good about having to reject a submission or getting a submission rejected.

3. Clutter/Ease of use. When you sort your collection (in real life) or go to buy from a store you don't look for the Playstation Move Controller between Pirates of the Caribbean and Power Gig.

Software (Physical) [Really doesn't relate at all but kind of needed to be here since i touched on the other 2... i guess.]
1. (PC specific)When digital started taking off and many people who heavily supported digital joined the site we ensured that they could view/track their collection in a way that worked for them by differentiating Steam/GoG/Origin/Etc from a PC retail release. Joyous cries were heard from the highest mountain to the lowest... low spot.

2. We added the virtual consoles/PSN/XBLA/GoD/XBLI for multiple reasons. People wanted to track that they owned I MAED A GAM3 W1TH Z0MB1ES!!!1 but didn't want it mixed in with their 360 games. While it is a game you play on the 360 Microsoft went through great lengths to let people know their system plays Retail/Games On Demand, Arcade and Indie games. Nintendo had done the same thing with the Virtual Console and WiiWare differentiating them in their own market places.

3. Variants have been(and probably always will be) a major point of contention. There are those completely against them. Those in favor of all of them and those in favor of some but not others. To a certain extent this problem has been dealt with for the users most irritated by variants returning when searching. When you want to know how many games were released for a specific system and learn that there are 8 versions of the same game bloating the number because of a potentially minor difference i can see how that would be frustrating.

*Initially all digital games fell under the umbrella of the system they worked on. Due to the ever growing demand that Steam and XBLA and PSN get their own support separate a split was made. That ended up opening up the doors for every other digital market place/distribution network for every other console.
When searching for an item to for any reason you wouldn't have to wade through a list of options completely unrelated to your search.

It is also worth noting we have never received complaints for not enough information being available in a search but do get complaints from too much making the user wade through to find the specific item(s) they want.



Really what it comes down to is a database split is in the best effort of:
Giving each item the right kind of attention
Keeping the database clean
Keeping everyone happy
Cataloging everything to the best of our ability. This is "The Classic and Modern Video Games Database" after all. Not the Cart-based only, CD-based only, Digital only, Hardware only, System/Company/Region specific Database.

If you'd like a proper response...
Everything you're "complaining" about is due to limitations with our tools and page design, NOT the database. Dave, in his infinite wisdom, made a separate database to track hardware as he felt that it was the simplest thing to do to allow for a separate page template and completely expand upon it. However, he didn't do a proper split, it still has ties to the Software database, and it's causing problems. I'd rather NOT extend the problem and instead correct it at its source, but that leads into the extremely large refactor of the database that I've been poking at for months, trying to find a simpler solution rather than forcing our entire staff to deal with it. Extending the hack that is the hardware database with the digital database would work at first, but it will eventually cause performance problems as it grows.

If we wanted to do this now, it would only require a day or two's worth of work to get up and running in a rudimentary way. However, we would need to do something to address the performance problem in a year or two or risk significant problems later on down the line.

EDIT: For what its worth, proper "filters" and "shortcuts" would fix 75% of the complaints that I've heard from people on the site.

[Izret101]

I think the source of all the problems has and will continue to be the forced integration of "databases" that should be fully separated.

It really seems that saying there is a hardware and a software database is like saying there is a 360 and a PS3 database. The only major difference is a little ID in the RFG ID system.

Even if i select Hardware on the search menu it isn't like it removes all non-hardware related items from the other search options.
No mater how many times i search for it i am never going to find a piece of hardware that is in the Action/Adventure genre.

[Shadow Kisuragi]

That's what I'm getting at about the broken filters - "Genre", in the backend, is referred to as "Type" in both databases. Splitting the databases may help for what you see visually, but it's something that can (and should) be fixed without causing other problems. The usability problems you're encountering are usability problems that aren't related to a database split.

For now, considering that we've seen how quickly I've jumped on this after we discussed it a year or two ago, I'll create the separate pages and make the appropriate changes, though at this point I may as well make the Strategy Guide and Arcade databases if I'm going through the effort...

[techwizard]

after reading through all the long posts, i think i'm leaning more towards what i believe Shadow is saying. which is (i think) to keep everything in one database, but to have a way of visually separating them to users. so on the backend, nothing has really changed, but on the front end it'll look completely separate.

i see it possibly working on a way of:
1. go to database search
2. it asks you whether you want to search for hardware, physical software, or digital software.
3. it visually hides all fields unrelated to what your choice above was, so that to the user it looks like a completely different search page/database, even though on the back end it still processes things through the same place.

the only thing i don't quite understand still is why splitting the database is worse for performance, if your search filters say "search hardware only", wouldn't that mean the servers have to look through much less data to find what you want?

[Izret101]

^That is what i have been trying to get at all this time.

"Genre", in the backend, is referred to as "Type" in both databases.

I just can't understand the argument for having the ever increasing cluttered results and search functions if they are already 2 separate DBs they should be treated as such.
And luckily adding digital should be less of a hassle since we have done a good job of keeping all the info from being mixed in with physical releases already.

For now, considering that we've seen how quickly I've jumped on this after we discussed it a year or two ago, I'll create the separate pages and make the appropriate changes, though at this point I may as well make the Strategy Guide and Arcade databases if I'm going through the effort...

Sounds good! Maybe instead of being just strat guides though it could include more mags like Nintendo Powers, Tips & Tricks, Sega Visions, etc, etc
Maybe even some other printed materials?

[techwizard]

after reading through all the long posts, i think i'm leaning more towards what i believe Shadow is saying. which is (i think) to keep everything in one database, but to have a way of visually separating them to users. so on the backend, nothing has really changed, but on the front end it'll look completely separate.

i see it possibly working on a way of:
1. go to database search
2. it asks you whether you want to search for hardware, physical software, or digital software.
3. it visually hides all fields unrelated to what your choice above was, so that to the user it looks like a completely different search page/database, even though on the back end it still processes things through the same place.

the only thing i don't quite understand still is why splitting the database is worse for performance, if your search filters say "search hardware only", wouldn't that mean the servers have to look through much less data to find what you want?

The problem is if you want to search everything.  I want to personally be able to search both digital and physical at the same time if I'm searching for something to see what platforms/distributions it's on.  It's less useful if I have to do two separate searches.

I don't think mysql supports cross database joins and queries, so if I wanted to query digital games in database D and physical games in database P in one fell shot (this is on development level, not on user level), then you have to run two separate queries, get the results from both back into memory, and then sort them in memory instead of using the mysql sort logic which is going to be multitudes faster on large data sets. 

ahh see that's the explanation i was missing, that makes sense.

@izret sorry i thought you were in the "split the databases in two" camp, it's all very confusing i think i understand more of what everyone is saying now though.

[Izret101]

I am
Added a couple extra words to help clarify.

While the argument of wanting to search to see everything does have validity it IMO is what the tool is least used for and will continue to be least used for in the future.

On the rare instances someone is going to be searching to see how many platforms / distribution services i think having to search in each of the databases for those specific instances would be less of a hassle than having to comb through all the "systems" / "genres" / unrelated results on a daily basis.

Personal example:
As someone who is only buying games from a specific series at the moment i use the database to search for just Mortal Kombat. I do i get the benefit of seeing every system/region it has been released for. In addition to the PSN, Steam, etc releases. While unlike some others i DO support digital I don't want those results returning in my searches.

I am not going to jump on eBay and find digital copies of those games, i'm not going to talk to some guy in Europe and see if he can find a copy of MK Komplete for Steam at GAME.
So i keep scrolling past them and have to readjust my count every time a new download is added.

We have multiple users with "junk" folders so they can stop things coming up in their searches. All that does is skew the site statistics. Users having to create "junk" folders to force better search results should be really telling that there is a problem with how things work now.

[Shadow Kisuragi]

...just to clarify what you expect:
Current Behavior:
Search Software, get Software results.
Search all, get all results.

Expected Behavior:
Search Software, get Software results (minus Downloadable Media Format)
Search Digital, get Software results with ONLY Downloadable Media Format
Search all, get all results.

Is this correct?

[Shadow Kisuragi]

Zthun, he's referring to tables, not databases. We currently have a table for games and a table for hardware under rfgene2_games.

[flyingclimber]

Are the db schemas published anywhere for us to peruse?

[Shadow Kisuragi]

Surprised so many people are actually interested in the site layout... To be honest, I could always use a hand, especially on the UI side, in getting the site up to modern standards. The PHP 5.5 upgrade (from 4.0) broke quite a few things that we STILL haven't fixed.

In terms of how it's laid out, Zthun's example is pretty close:
[rfgene2]
    [rfgene2_forum]
        [....]
    [rfgene2_games]
        [...]
        [games]
        [hardware]
        [...]

Worth noting that the DB schema, and many of the columns, are in sore need of a refactor. Keys aren't properly used in many instances, which causes a lot of problems - especially with strings.

[techwizard]

Surprised so many people are actually interested in the site layout...

for me i just like hearing technical explanations of why things do or don't work, it's much nicer than a generic "sorry we can't do this" or "that's a bad idea" with no real explanation. transparency is awesome

To be honest, I could always use a hand, especially on the UI side, in getting the site up to modern standards. The PHP 5.5 upgrade (from 4.0) broke quite a few things that we STILL haven't fixed.

i wouldn't mind helping out, but i might be a bit slow at first because i would have to do some research to learn more about php. i could definitely help with hunting down/fixing minor UI bugs, typos, etc around the site.