Thank goodness for DVRs
This Christmas we were treated to the second Disney TV broadcast featuring the brand new Connect 3.0 tech that brings Made with Magic support into the home. We actually missed the broadcast but were able to record it with a DVR and play it back once we got back home.
Fortunately, the show queued up in the Shop Disney Parks app was still the Christmas show. So I was able to get the Connect stuff to work and do a little more sleuthing into how all this works.
Correcting previous crazy ideas
It turns out all the show data isn't contained in the video
After the first show, I came up with the idea that maybe all the data necessary for the show was contained in the actual video. The reasoning behind that idea was due to the fact that the whole show would replay nicely when the DVR recording was played back - even if the phone receiving the audio was completely disconnected from the Internet so it couldn't receive any data over the wire. That idea isn't far fetched. Check out this link to see actual Disney research into using data embedded in audio streams.
OK. So the idea isn't far fetched. But it also doesn't appear to be right. Once the phone app switched over to promoting the Christmas show, playing back the recorded video didn't do squat. Based on that observation, I'm going to revise my theory to be that the meat of the show's data is uploaded to the phone app and the show audio serves to trigger play from stored info on the phone. In support of these ideas, I did note that I can pause the video and the phone animations and Made with Magic effects happily continue on until what would be the anticipated end of that particular segment. Similarly, I noted last time that the phone can be completely cut off from any Internet/Cellular connection and still respond to the audio signals from the video recording. The necessary data doesn't seem to stream upon demand.
OK, so the new Ear Hats DO echo the Bluetooth codes to legacy devices
At the beginning of the previous show I put the new Connect 3.0 Ear Hat next to an older Ear Hat and failed to see any coordination of signals between the two during the show. So I concluded that the new hats don't play nice with the older stuff. Turns out that's not true. I happened to prop up a Paintbrush between the ears of the new hat during the recent play back and, lo and behold, they sync'd during the show! You can plainly see such interaction with a legacy device in the video at the top of the page. What gives? Well, the echoed signal is fairly weak. Last time I inadvertently aligned the new and old ear hats such that the IR receivers were as far apart as possible and the height difference between the hats may have helped block the IR signal from reaching the older hat's receiver. This time, I put the two ears with the emitter and receivers right next to each other and it worked. I'm guessing the emitter's range is less than 4-5 feet and easily blocked by obstacles in the path.
This is sort of good news for folks with the older stuff. As long as there's at least one new Connect 3.0 hat and the older gear is close enough to the new hat, then the older gear will glow along, too. Now, if you watched the video above, you may note a bit of a lag in response by the older ear hat. There could be a couple of reasons for that. Let's take a look.
Connect 3.0 Made with Magic codes
Let's start on the Bluetooth side of the fence
I used a BLE sniffer from Adafruit to monitor the show and dump results into Wireshark (you want all 171 MB of the MwM packets? Get it here). Picking through the results it was easy to find the goods this time. Here's a representative packet:
No. Time Source Destination Protocol Length Info
2794 741.691319000 Slave Master LE LL 62 ADV_IND
Frame 2794: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) on interface 0
Interface id: 0 (\\.\pipe\wireshark_nordic_ble)
Encapsulation type: USER 10 (55)
Arrival Time: Dec 31, 1969 16:17:36.009725000 Pacific Standard Time
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1056.009725000 seconds
[Time delta from previous captured frame: 0.022467000 seconds]
[Time delta from previous displayed frame: 0.022467000 seconds]
[Time since reference or first frame: 741.691319000 seconds]
Frame Number: 2794
Frame Length: 62 bytes (496 bits)
Capture Length: 62 bytes (496 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: nordic_ble:btle:btcommon]
Nordic BLE sniffer meta
board: 21
uart packet counter: 22238
flags: 0x01
.... .0.. = encrypted: No
.... ..0. = direction: Slave -> Master
.... ...1 = CRC: OK
channel: 38
RSSI (dBm): -44
delta time (us end to start): 366
delta time (us start to start): 886
Bluetooth Low Energy Link Layer
Access Address: 0x8e89bed6
Packet Header: 0x2440 (PDU Type: ADV_IND, TxAdd=false, RxAdd=false)
..00 .... = RFU: 0
.1.. .... = Randomized Tx Address: True
...0 .... = Reserved: False
.... 0000 = PDU Type: ADV_IND (0x00)
00.. .... = RFU: 0
..10 0100 = Length: 36
Advertising Address: 50:4f:80:ef:4f:42 (50:4f:80:ef:4f:42)
Advertising Data
Flags
Length: 2
Type: Flags (0x01)
000. .... = Reserved: 0x00
...1 .... = Simultaneous LE and BR/EDR to Same Device Capable (Host): true (0x01)
.... 1... = Simultaneous LE and BR/EDR to Same Device Capable (Controller): true (0x01)
.... .0.. = BR/EDR Not Supported: false (0x00)
.... ..1. = LE General Discoverable Mode: true (0x01)
.... ...0 = LE Limited Discoverable Mode: false (0x00)
128-bit Service Class UUIDs
Length: 17
Type: 128-bit Service Class UUIDs (0x07)
Custom UUID: 707070707024676958f04803d0110400
Device Name: MWMGLOW
Length: 8
Type: Device Name (0x09)
Device Name: MWMGLOW
CRC: 0xf30be4
[Expert Info (Chat/Protocol): correct]
[correct]
[Severity level: Chat]
[Group: Protocol]
0000 15 06 37 01 de 56 06 0a 01 26 2c 00 00 6e 01 00 ..7..V...&,..n..
0010 00 d6 be 89 8e 40 24 42 4f ef 80 4f 50 02 01 1a .....@$BO..OP...
0020 11 07 70 70 70 70 70 24 67 69 58 f0 48 03 d0 11 ..ppppp$giX.H...
0030 04 00 08 09 4d 57 4d 47 4c 4f 57 cf d0 27 ....MWMGLOW..'
That part, Device Name: MWMGLOW, is a dead giveaway that this is where the action is. The 128-bit Service Class UUID has been hijacked to transmit the Made with Magic codes. Looks like the MwM phrases are left-padded with 0x70 bytes if the full 16 bytes aren't needed for any particular transmission. That's different.
Note that the codes are being sent as advertising packets. Nowhere did I note any actual connections taking place. I guess the advantage here is that a lot more targets may be serviced this way than if actual connections were made. The downside to this approach may be that some synchronization is sacrificed. Minimum delay between advertising packets is 20msec+random delay. The packets first appear on channel 37, then 38 and 39. They're then repeated on the same channels in that order. Consequently, I calculate that there could be a 100msec+ delay between the first transmission on channel 37 and the last one on channel 39. That's about a 10 Hz frequency that the eye can easily detect. Add to that, the effect of the delay from rebroadcasting to legacy devices and potential for lags becomes even greater. At 2400 baud and 10 bits/byte, we can push out 240 bytes/sec or about 1 byte every 4.2msec. We need to send 18 bytes out each time, taking 75 msec to do so. That's roughly a 13 Hz frequency, which is also likely to be noticeable. A worst case scenario is a 175+ msec difference in responses. Hmmm.
Now, let's check the IR recordings of the echoed codes
Now that I knew the codes are, indeed, echoed to the the legacy devices, I tucked an Arduino recorder right underneath the emitting ear of the new Ear Hat to see what I could catch. You can get the full recording file by clicking this link. Here's a snippet from that recording:
0005C87E: 9F 70 70 70 70 70 24 48 83 D0 0E FF D1 41 06 07 51 FF 00
0005C99C: 9F 70 70 70 70 70 24 48 83 D0 0E FF D1 41 06 07 14 D9 00
0005F4AC: 9F 70 70 70 70 70 24 61 6E 58 F0 48 04 D0 42 04 14 F1 00
00061E26: 9F 70 70 70 70 24 64 6A D0 32 F0 D2 50 90 40 00 09 F1
00061F51: 9F 70 70 70 70 24 64 6A D0 32 F0 D2 41 07 02 40 00 09 00
0006207B: 9F 70 70 70 70 24 64 6A D0 32 F0 D2 41 07 02 40 00 09 00
000634B2: 9F 70 70 70 70 24 64 68 48 86 FA 24 60 6F 48 86 00 0A 00
000635D3: 9F 70 70 70 70 24 64 68 48 86 FA 24 60 6F 48 86 00 0A 00
0006411E: 9F 70 70 24 64 68 48 86 FA 24 60 6F CA FD 24 00 38 0A
000642A2: 9F 70 70 24 64 68 48 86 FA 24 60 6F 48 86 FD 12 38 0A
The first things that stick out: the left-padding with 0x70 chars is transmitted, making all phrases the maximal allowed length and the replicates, if they are replicates, are often not that good. And even the replicates have quite a lag in timing:
00061F51: 9F 70 70 70 70 24 64 6A D0 32 F0 D2 41 07 02 40 00 09 00
0006207B: 9F 70 70 70 70 24 64 6A D0 32 F0 D2 41 07 02 40 00 09 00
That's about a 298 msec difference. If that's the best precision that can be expected from the new tech, then the tightly synchronized shows we've become accustomed to probably aren't possible going forward.
Another thing I don't see is the whole series of leading 0xFX delays. These commands appear intended to be executed immediately. Hmmm. Wonder if that's what causes the new hats to choke on the playback of older show recordings that I described in a previous post? Guess I'll have to test that one of these days.
Ideas for making an IR blaster for legacy devices
I can imagine a couple of different projects to get at the Made with Magic codes for these shows. Probably the easiest would be to monitor the Bluetooth output, dump it from Wireshark into an XML file and extract the data using a custom program to output the customary MwM file format used by most of our existing MwM players. That shouldn't be all that difficult. As a result, we'd have a nice video of the show and the codes to blast out to our MwM gear. And we'd have something that won't evaporate when the Shop Parks app switches to a newer show. Nice. This is something that should probably be done in the not-too-distant future.
Taking it a step further, we could use a Bluetooth enabled micro to catch the feed in real time, extract the MwM codes, add the 0x9F prefix, calc the checksum and blast out the IR codes to the legacy gear. I'm guessing that approach would appeal most to folks without DVRs. Something to think about but not high on my list of priorities.
Circling back to how the setup may work
I've run across a couple of posts about how audio may be used to transmit info. One Disney patent involves both fingerprinting and watermarking tech. So we could be seeing either or both. Maybe one clue comes from what appears to be an ad gone wrong. Check out the video. An ad for Frozen starts off and, in the middle of it, the phone and ears start up a little routine which continues on through a couple of other commercials! That can't be right.
So what are we looking at? Clearly, that wasn't meant to happen. The frozen commercial was way too short for that animation. I'm guessing that some part of that Frozen commercial tripped an audio fingerprint that started up that animation. That would be my best guess. Time will tell if I'm on the right track, or not. If you've got insights, do share.