Yeah, that would be really useful.  Most of my paragraphs are really well suited to collapsing everything after the first sentence of each paragraph.


The polling rate isn't configurable - the PIUIO driver runs as fast as it possibly can given the hardware and USB drivers and can't be told to do anything else.

Dedicabs have low polling rates because they use USB 1.1 drivers, even though the computer has USB 2.0 ports.  I can't remember if it was shakesoda or vyhd who pointed out to me that, in 2004, Linux USB 2.0 drivers were still unreliable, since it takes a little while for open source drivers to mature and USB 2.0 was very new at the time, so Roxor's decision made sense.  They compensated for the old drivers by coding the game to poll as often and as fast as it could, and added JWA to compensate when that still wasn't enough.

---- TL;DR summary at bottom ----

JWA can be set to whatever - you can even make it negative and shrink the windows.  It's worth noting that the OpenITG guys have conducted placebo tests where they tell people they're changing the sync (by the 6ms r8-to-r16 difference) and actually do nothing; players commented that the placebo sync feels more like r16 - subjectivity and standard deviation in a player's scores combined can do more more than a 6 ms sync change.  Since the polling rate is more a function of stabilizing the sync than widening the windows once you get above 100 Hz, I think the same would apply here, so players probably couldn't reliably tell JWA=0 at 160 Hz from JWA=0 at 480 Hz, since that's an offset of 2 ms on average.

Furthermore, when you consider that 43 ms at 160 Hz is 6.88 samples, you find that the worst case window is 37.5 ms (6 samples, giving a window of about -16 ms to +21.5 ms) but in in 88% of cases, the game will give you 7 samples, or 43.75 ms (-21.5 ms to +22.5 ms, -20 ms to +23.75 ms, and -16.5 ms to +27.25 ms are all possible).  On the other hand, at 480 Hz, you get a wost case of 20 samples, or 41.7 ms (roughly -19 ms to +21.5 ms), and 64% of the time, you get 21 samples, or 43.75 ms (just like 7 samples at 160 Hz, but the most skewed window possible is only -19.5 ms to +23.5 ms).  It'll play like an upgrade with really nice pads, except, instead of encouraging you to step off beat to take advantage of the window skew, and having every 8th arrow be significantly tighter with a 1 ms skew, you get windows that are never skewed more than 1 ms, no random timing spikes (the tighter windows are still close to correct), and only 3 out of 5 - instead of 7 out of 8 - arrows have the unintended 0.75 ms padding.  I'm not going to do the math for 480 Hz and JWA=0.15, since that's clearly an inappropriate configuration (@Tom: fix your machine).

Now then, the current setup, JWA=0.0015 at ~90 Hz, has a total window of 46 ms, or 4.14 samples.  This means 86% of steps get 4 samples, or 44.4 ms (for example -21.6 ms to +23 ms, or, at the heaviest skew, -12 ms to +32.4 ms) and 14% get 5 samples or 55.6 ms (that's so wide that there's no reason to do the calculations).  As long as you take advantage of the skew for a 4 sample window by centering yourself about 6 ms behind the theoretical center (which the global offset is meant to do for you), you get to enjoy windows that are never smaller than the proper 43 ms, with the skew never hurting you more than the worst case for an upgrade, and roughly every seventh step is obscenely easy.  So, while the higher polling rate will fix the worst case where one edge is only 17 ms from the adjusted center and ensure than a 1 ms center adjustment never gives you a worse case than 20 ms, the higher polling rate will instead ensure that the timing window can never be wider than the narrowest timing window the computer currently gives.  So, when it comes to stepping against the skew, the worst case with the new computer is no where near as bad, nor will it happen as often, but in every other way, the judging will be less generous.

TL;DR: JWA=0 at 480 Hz is fine.  We might end up needing to change the global sync by about 2 ms, though.

I think the solution to all of this is fairly obvious.  We just need to set the polling rate and JWA to the rate that's set by the National Institute of Standards and Technology.

http://www.nist.gov/ITG-portal.cfm

Oh wait.

------------------------------------------------

Honestly, it's not like the machine was "correct" before the new computer or will be "correct" afterward.  Are the scores within reasonable bounds of the TONS of different machines that exist out there?  Is the machine still enjoyable to play?

If someone has some specific polling rate or JWA to make a solid argument for, do that.  But Tony is the only one making logical (and thorough) arguments for the choices that he's making to maintain the machine.  I think he's doing a fantastic job with all of his research and efforts; and while the machine might be a little different, change is not necessarily a bad thing.

Tony's doing us a service by upgrading the machine's hardware at the cost of lots of his own time.  Your concerns may be legitimate, but I take issue with you complaining without asking if anything can be done about it first.  Please try to be constructive when you raise issues like this, and show a little appreciation for the work that's being put into the project as a whole.

Dude, I appreciate the level of detail in your posts, but it makes me wish our forum had a /spoiler or /collapse feature for large chunks of detailed text like that.

The polling rate is being offset by JudgeWindowAdd going to 0, per vyhd.

A polling rate that high will get rid of the occasional issue with a sample lining up as horribly as possible, which is what JudgeWindowAdd was meant to do on the old computer.  If JWA=0.0015 at 80-100 Hz hasn't been enough to prevent significant changes in your scores, then you manage to play with astonishing precision, but are consistently 20ms off-beat.  I'd think anyone that precise would be able to get themselves on beat though, so I doubt that's happening.

I need to check with Vyhd, but given that r16 (which bumped the polling rate) also included a change to the global offset, I have a feeling that polling samples are rounded in the same direction, rather than inward - one side of the window shrinks while the other grows, effectively changing the sync (by an amount that varies with each arrow).  The r16 offset changed by 0.006, which happens to be on the order of the difference in sample sizes with and without the kernel hack (it's about one half of the difference between samples at 40Hz and 80Hz and close to the difference between samples at 60 Hz and 100 Hz); this could well just be coincidence, which is why I will ask vyhd when I get a chance, since he's likely to actually know.  In that case, the higher polling rate wouldn't effectively widen the timing in any way, it would just make the center of the window more likely to line up with the intended timing.  And then, having JWA=0 thrown in, the timing window actually shrinks, to compensate for the window drifting less.

Assuming instead that the rounding errors only noticeably affect one side of the window (I'll call it the front side), with the back side staying very consistent, the lower polling rate paired with JWA=0.0015 advantages playing toward the back side of the window, since that side will not only be more consistent, but will also still have the full 1.5 ms JWA padding applied.

The higher polling rate is only an advantage if both sides of the window round inward.  Assuming that's true, then with JWA=0.0015 (a total Fantastic window of +\-23 = 46 ms) and 100 Hz samples (conservative, since that's the best case), the worst case would be 3 Fantastic samples per step (for example, samples at -24 to -14 and 16+ to 26 would be dropped, leaving a window of 30 ms, or 3 samples), with the best case being 4 samples which give a slightly asymmetric window.  The issue here is that the worst case of 3 samples would happen over 1/3 of the time, so we'd have had to have gotten used to dealing with windows that are essentially +/- 14 ms, instead of the theoretical +/- 23 ms, or even the best case 4 sample windows that would be more like +/- 19 ms on average (due to the slight asymmetry).  Factor that the polling rate could drop (the polling rate varies constantly and rapidly) to 80 Hz (12.5 ms windows), and would get arrows that spontaneously have windows as tight as +/- 12 ms.  Any player who could achieve any degree of consistency across multiple tries on the same chart (the timing window for each arrow would vary from play to play, so they'd basically be able to play the entire chart as if the Fantastic window were +/-14 ms, with the occasional 12 ms window most likely giving the an Excellent).  Given these kinds of numbers, I definitely find it safe to assume that the windows don't round inward, but I'd be glad to pick some charts that I can consistently SDE and see if the new computer starts handing me quads in an attempt to check these numbers.

That brings us to one of the previous two models - the whole window shifts or the front side shrinks while the back side is fixed.  The back side can't be fixed, since that requires the samples to be precise enough to not vary the window by more than a fraction of a millisecond, so we're back to my assumption that the whole window shifts by some amount and may occasionally shrink slightly when things really line up poorly.  In that case, cutting out the 1.5 ms of window padding by removing JWA should more or less balance out and the theoretical concerns are moot.  I'll be glad to take back my stance if someone gets a half dozen new quads in one day on the new computer, but for now, there's no reason to theorize that scores will show any statistically significant changes, since for every step hurt by a front side rounding error, one is likely helped by a back side rounding error.

I only played a couple songs on the new computer before I pulled it back out, so I don't have a large enough sample to make any strong claims, but it was not noticeably easier - the only score I remember offhand is Romeo's Dead, on which I got 17 one day after getting 9.

To be fair, some songs will be significantly easier with the new hardware.  Those songs would be ones with 640x480 videos, like Less Than Three.  Or ones that crash the machine sometimes, like Sweet Donuts.

I didn't see you guys bitching about the new computer when it was announced.  And even if one of you didn't have access to the internet when the new computer was announced, you could have bitched about the new computer much sooner than when it's about to get put in.

Because my DDR/ITG skills have been so stagnant over the past three months.

So basically scores are going to improve by virtue of software and hardware changes as opposed to skill again. Awesome.

Got everything hooked up and running, but had to swap the old computer back in due to some configuration issues.  Most of it is small stuff, USB is the only major issue.  If I can get everything in working order, I might stop by again this evening.

I am happy to note the new computer has a polling rate which varies between 460 and 500 Hz, so the polling rate should never again hurt anyone's scores.

Alright, time to try this again.

Day 1 was a bust - the PIUIO card gave me some trouble.  I'm still trying to figure out if I had things plugged into the wrong place or backwards or what.  Gathering information today, giving it another try tomorrow morning.  I'm going to shoot for 9 am, but we'll see how that goes.  Definitely no later than 10 am.

I'll be heading down there starting right now.

In a couple minutes, I'll be loading everything into my car and heading to Acme to start the install.  Everything looks good so far, so hopefully there will be no new issues once the new computer is wired into the machine.

Well, the test session is tomorrow morning.  If plan to come help out, make sure to remember your USB drive - I've only been able to do a bare minimum of USB testing without having the actual hub connected, and I'll only have a pair of drives with me, which is far from enough to do a thorough test run.

I've had a busy week, so progress has been slow.
- I've done about a quarter of the work need to fix scores, and I honestly won't be done in time.  I'll be copying the current machine stats.xml to the new computer tomorrow morning, and fixing the file up as soon as I'm done with the cleanup code.  High scores for any files that are in different packs that they used to be or that have been deleted because they are duplicates from another pack will disappear for a few days, but will come back once I finish the cleanup program.  The program will merge scores as appropriate, so so new scores will be be merged with the old - no scores from the new or old computers will disappear... well, if I need to run multiple test sessions, only scores from the last test session be kept, but that's the only possible loss.

Theme work has been at a relative standstill as well, so I'll leave the current Chromatic version in place and wrap up the fixed version soon.  Setting up the correct boot sequence is the only critical (i.e. keeping me from installing the new computer) work remaining, and I'll be wrapping that up tonight.

Zim 1/Young Man
Awesome chart/song, I vote to keep it.

Profile keys are on the new computer, Mute/Zim have been updated to reflect the songs posted so far (note to sfxazure: Zim 4 - In and Out Of Love does not seem to exist).

Remaining work:
- Keep trimming Mute/Zim as people post about them.
- ITG3 Chromatic (about 30% done)
- Boot sequence (about 50% done)
- Fixing scores (have not started, but it shouldn't be too much work)

This is somewhat separate from the song request thread.  This is people voting for stuff that they don't want me to delete from existing packs, instead of proposing individual things to be added.  I felt it made more sense to get people to aggregate this information in the general maintenance thread, since it would create a lot of confusion in the requests thread.

People don't like bumping threads, and it's also more pertinent since it's less about adding songs and more about keeping songs.

We did have a thread for song requests...

I vote for this too lawl

Off the top of my head, here's a list of Zim songs to keep.

I'll definitely be there during test weekend.

By the way, off the top of my head, I also want to keep
Mute Sims 5/NoN-Fiction Story!
Mute Sims 5/Papi (Jankenpop mix)

After talking to vyhd, I've figured out why I'm missing the profile signature keys and will be getting them off the Acme machine next time I'm there.

The remaining work on the new computer is:
- Profile keys
- Edit the boot sequence to make ITG load by default.
- Trim Mute / Zim packs, add Gerrak's pack.
- Finishing the ITG3 / Chromatic hybrid theme (the skin will be almost unchanged, but we'll have a few minor ITG3 / OpenITG modifiers and some very important OpenITG features that are missing from the operator menu).
- Live testing at Acme.
- Code up something which can edit the machine stats so that scores for songs which are being moved around will be maintained (this includes merging the two high score lists for duplicates).  If I code something to do it, I don't have to do it by hand, which means I can do it at Acme and we do not lose any scores - if I do it by hand, the fixed copy will be a few days old and we'll lose a few days worth of scores.

If everything lines up perfectly and I'm not particularly busy this week, the new computer could be ready by this weekend, but I'm not counting on it.  That said, I'm looking to run a test session next weekend (27th-28th) instead.  I know PAX is that weekend, but I don't believe everyone here is going, plus I really only need 2-4 people to help me test, and having too many will make it hard to coordinate everything, since I made need to interrupt people or tell them specific things to play.  I won't turn away anyone who shows up to help, but if you have other things to do (like being at PAX), you don't need to go out of your way to show up that day.

The tentative plan is for me to get there before 11am (probably around 10am, but we'll see what happens) on Saturday (the 27th) to set up, and anyone who wants to help test things should show up at 11am (I need time for setup and to do the final work to get USB drives working through the hub).  Around 1pm, I'll decide whether the computer should be considered stable or not and swap the old computer back in if necessary.  If there are serious problems, I will try to fix them that day and run a second test session Sunday morning.

Once I'm sure the computer will be ready for the 27th, I'll start a separate thread on the events board.

I have Bill's permission to put the machine on freeplay during computer test sessions.

For one reason or another, I don't have Mute 5 on there - I'll download it when I get a chance and post the song list once I have it.

Oh yeah, DDR Illini with duplicates removed is 36 songs, which is a good size.  H1N1 without duplicates is just over 90 songs (there were a lot of duplicates against Tachyon Alpha and a few against other packs), which is still pretty sizable, but still a lot better than it was - I might still want to look into trimming it down a bit, but one larger pack is going to contribute less confusion that a large group of small similarly named packs (Mute, Zim), so I'm not all that worried about trimming H1N1 for quality at the moment - if there is a blatantly bad chart, let me know and I'll take a look at it, but I'm not doing anything elaborate.