Custom Firmware on the OP-1

First things first. A disclaimer:

Dear Teenage Engineering: I love your company, I love the OP-1. Your design work is incredible and you seem like one of the nicest companies I know. So please understand that I do not in any way, shape or form want to harm you. I simply like to tinker and play with my hardware and I love reversing stuff and finding out how things work. Since this research may very well lead to custom firmware with your IP in it, I will not release it without your blessing. If you want to get in touch, send me an email to
tabascoeye AT gmail DOT com.


Now that this is out of the way, let's get into it.

I stumbled into all this after getting my OP-1 about 2 years ago. Looking at firmwares is kind of a hobby, so naturally when a filetype ".op1" comes along, I'm interested in taking it apart.

1) OP-1 update file contents

The update files that you can download from teenage engineering actually are just a compressed file containing more files. The format is called LZMA and is a compression algorithm. To make sure that the file arrived on the OP-1 when you transferred it, a 4Byte checksum (CRC32) is added infront of the compressed file.
That is usual practice with files in embedded systems.

So if you strip those 4 Bytes, you can just decompress the contents. (Since I am a lazy bastard, I usually just let 'binwalk' do its thing on files I don't know)

The contents are [using the leaked Beta FW op1_076.op1]:
.
├── content
│   ├── audio
│   │   ├── drum
│   │   │   ├── drum_0.raw
│   │   │   ├── drum_1.raw
│   │   │   ├── drum_2.raw
│   │   │   ├── drum_3.raw
│   │   │   ├── drum_4.raw
│   │   │   ├── drum_5.raw
│   │   │   ├── drum_6.raw
│   │   │   └── drum_7.raw
│   │   ├── factory_drum
│   │   │   ├── adm perc1.1.raw
│   │   │   ├── berlin.raw
│   │   │   ├── bread of sugar.raw
│   │   │   ├── deep africa.raw
│   │   │   ├── detroit.raw
│   │   │   ├── funky old.raw
│   │   │   ├── future tech.raw
│   │   │   ├── india.raw
│   │   │   ├── jazz.raw
│   │   │   ├── jeff-s 01.raw
│   │   │   ├── jeff-s 02.raw
│   │   │   ├── lenk.raw
│   │   │   ├── live-ish.raw
│   │   │   ├── neon-80s.raw
│   │   │   ├── reggae type.raw
│   │   │   └── steeezo r2.raw
│   │   ├── factory_synth
│   │   │   ├── bladerunner.raw
│   │   │   ├── creep dish.raw
│   │   │   ├── distort a log.raw
│   │   │   ├── engine rev.raw
│   │   │   ├── ghettobass.raw
│   │   │   ├── harm a chord.raw
│   │   │   ├── op303_s2.raw
│   │   │   ├── punchsub.raw
│   │   │   ├── rmk2 note.raw
│   │   │   └── subway bass.raw
│   │   ├── preset_drum
│   │   ├── preset_synth
│   │   ├── speech
│   │   │   └── op1patch.raw
│   │   └── synth
│   │       └── synth_7.raw
│   ├── display
│   │   ├── adsr.svg
│   │   ├── album.svg
│   │   ├── apetape.svg
│   │   ├── bendlfo.svg
│   │   ├── bode.svg
│   │   ├── clock.svg
│   │   ├── cls.svg
│   │   ├── colors.svg
│   │   ├── com.svg
│   │   ├── cranklfo.svg
│   │   ├── dbox.svg
│   │   ├── delay.svg
│   │   ├── drum2.svg
│   │   ├── drw.svg
│   │   ├── dsynth.svg
│   │   ├── duallfo.svg
│   │   ├── dynaenv.svg
│   │   ├── endless.svg
│   │   ├── eq.svg
│   │   ├── etchasketch.svg
│   │   ├── fm.svg
│   │   ├── ftwo.svg
│   │   ├── grid.svg
│   │   ├── help.svg
│   │   ├── id.svg
│   │   ├── in.svg
│   │   ├── iter.svg
│   │   ├── lander.svg
│   │   ├── lpc.svg
│   │   ├── master.svg
│   │   ├── micline.svg
│   │   ├── midilfo.svg
│   │   ├── midi.svg
│   │   ├── mixer.svg
│   │   ├── mllp.svg
│   │   ├── mmmf.svg
│   │   ├── octave.svg
│   │   ├── ok.svg
│   │   ├── opfont.svg
│   │   ├── pattern.svg
│   │   ├── pd.svg
│   │   ├── playmode.svg
│   │   ├── pls.svg
│   │   ├── presetbrowser.svg
│   │   ├── ptch.svg
│   │   ├── radio.svg
│   │   ├── reroutelfo.svg
│   │   ├── rndlfo.svg
│   │   ├── rymd.svg
│   │   ├── sampler.svg
│   │   ├── save.svg
│   │   ├── signalflow.svg
│   │   ├── simple.svg
│   │   ├── singlelfo.svg
│   │   ├── sketch.svg
│   │   ├── slump.svg
│   │   ├── st.svg
│   │   ├── subscreenhand.svg
│   │   ├── t10.svg
│   │   ├── tapeconfig.svg
│   │   ├── tape.svg
│   │   ├── tempo.svg
│   │   ├── tombola.svg
│   │   └── tune.svg
│   ├── kerntable.db
│   ├── op1.db
│   ├── op1_factory.db
│   └── tape.db
├── OP1_vdk.ldr
└── te-boot.ldr

10 directories, 107 files


«13456714

Comments

  • Thanks a lot! :)

    .raw is sound and .svg is a image vetor format, but how about the .db and .ldr files? What are those formats? 
  • .ldr, hmm. Maybe, and that would be funny and awsome at the same time, it is that: https://en.wikipedia.org/wiki/LDraw
  • .db should be some sort of database file
  • 2) The Display

    One of the oddest things about the OP-1 when I read about it first was the marketing notion of the "vector based display" since the OP-1 clearly has a pixel based OLED display.

    So in essence probably someone heard from the developers that they use "scalable vector graphics" (SVG) to create all the graphics.

    The fun part about SVG is that it is an XML based format that has concepts like "display=none", ids and groups. So they could likely split into the programmer group, writing the animations by using ids and switching "display=none" to "display=inline", and a designer/graphics group that could change the actual graphics without the need of re-programming anything.

    Btw: Most of the graphics were done with Adobe Illustrator CS4, some with CS3 and in the beta, there are files done with "17.1.0" which is probably Illustrator Creative Cloud.

  • edited July 2016
    3) The .ldr files

    The OP-1 works with a BF-524 as main CPU. That is a "BlackFin" DSP chip made by Analog Devices. It is also used in digital storage oscilloscopes because of its DSP capabilities.

    The blackfin uses a weird format for files and documentation isn't that easy to find. Essentially, the LDR (or "loader") format is what the CPU expects as download format. It consists of "blocks" of data, each with a header and there is a bit of "compression" involved (namely there are headers which say "the following X Bytes are all 0") thus when loading that file, the CPU fills its memory according to the instructions in the loader file.

    According to AD documents, the file that is represented by all the blocks in the LDR file is a "DXE" file. I found some mentions about DXE being a lot like ELF, which says a lot and nothing at the same time.

    So essentially, to _really_ run your own code on the OP-1 you would have to:

    1. write a decoder software to turn the LDR file back into DXE
    2. find out the format of DXE
    3. write a disassembler for blackfin (or hope that e.g. radare works with it)
    4. analyze the codeflow and delete/insert parts where needed
    5. turn the DXE file into LDR again
    6. re-package the whole OP-1 firmware update package
    7. upload! \o/
    and yes. That is about as much pain as it sounds.
    At first I thought that the OP-1 would run ucLinux on there because there are some tools and eval kits for the BlackFin running that. But there is no reference whatsoever to Linux in there, so the theory of it using VDK (the kernel from VisualDSP) is much more likely
  • @TabascoEye so how long until I can install your hotrodded OS? Kind of like Jailbreak for the iphone. LOL
  • 4) inner workings of the firmware

    The firmware uses quite some known opensource software:

    • yaffs2 - a file system for flash chips (the memory of the OP-1, noted as "grandmaster flash" on the PCB)
    • libaiff-5.0 - a library to mess with the aiff files used for audio
    • sqlite 3 - a relational database
    • box2d - a physics engine (most likely for things like tombola and the chopper game)
    • libjson - a helper library to parse the JSON format
    • probably more...
    I found out most of this by just getting every readable character (ASCII) from the .ldr file. In Linux there is the "strings" command for that.

    The usage of SQLite databases and SVGs is pretty ingenous as I mentioned because they could change large parts of the OP-1 without the need of recompiling all the C++ code and run it through the whole toolchain. Instead, they could just exchange SVG files or write other values in the SQLite DBs to change the behavior of the synths etc.

  • 5) the .db files

    maybe the funniest moment was when I checked the database files which are in the update package in a SQLite browser.

    There is "kerntable.db" which consist of the columns "a","b" and "kerning".
    You can be sure you are dealing with designers when there is a database that controls the kerning between each symbol on there :o))
    Did I mention I love that company?

    kerning

    so apart from this fun file, there is "op1_factory.db" which seems to be the key file in doing a "factory reset". And that contains this very interesting table:


    OP1_SQLite

    notice the "id" column. So there are some values missing: 1, 2 and 6....

  • 6) Moving on...

    So now there are 3 areas of major work:

    1. trying out the correct values of the fx_type database table
      1. which type fits which id (is the order even important? it seems so based on my experiments)
      2. which default_params are needed for the missing fx? (this is a major pain in the butt to figure out because of the time it takes to create an FW, download it, try, fail, factory reset, back to square one)
      3. find out if the fx even works with the current firmware. maybe its broken or barely usable? Theres probably a good reason for taking them out...
    2. trying to find out the workings of the SVG files to create some nice custom graphics (as indicated in my other post.. but.... wait for it ....)

      and finally:
    3. the cown jewels: creating all the tools necessary to crack the actual firmware open, figure out how it works and mess with it until something new comes out (usually the "hello world" of custom firmware on devices is getting "DOOM" to run..... ... .....)
    So thats the current state and I'll continue to work on it when I have some spare time.
    In the mean time I will give you a teaser of what I have in mind when talking about "custom graphics" :o)
  • ...chorus, lpC and tape ape!? :D And that filter fossile? 
  • dropping the bomb

    image
  • dropping the bomb

    image
    Haha F%ck yeah!

  • you got my attention o_o
  • Hahahahaha <3
  • @TabascoEye Ahaha excellent mate.
    Hope you manage to get the equivalent sonicwise, that would be awesome :D

  • seconded .db is sqlite. 
    some info on the OP1 chunk in AIF presets here. of course if you wanted to generate a ton of patches, it is totally possible.
  • man some awesome fun could be had editing the SVG's throughout the OS :)

  • Yeah it'd be really interesting to develop an alternate graphics pack :D
  • The blackfin uses a weird format for files and documentation isn't that easy to find. 

    ftp.analog.com and maybe pub/dsp/blackfin and then there's more stuff in pub/dsp/tools and some juicy stuff in e.g.pub/dsp/210xx/code_examples/65l-ezlab or apps_handbook, admittedly for a different chip but  , additionally you can find VisualDSP on the Analog Devices site, as well as a library of software modules and more software and tools for the blackfin. i'm sure some of the binary output of these will give you recognisable patterns to munch on.. the official dev kit is $$$

  • Just so nobody gets their hopes up in this thread: I will not post any complete firmware here as that would contain Teenage Engineerings intellectual property and I would be redistributing it, which is copyright infringement (see cyanogen mod for android).

    What I can do however is post instructions on how you can create your own firmware.
    We can probably share mod instructions and maybe even modified/new SVG files and .db files, but full firmwares will be a no-no.
  • @tabascoeye dang... This tech stuff is waaay beyond me to even follow instructions! Hopefully if some cool custom stuff gets made by users it will find its way to TE/official OS at some point. Guess you guys could maybe even earn a little £ potentially if were able to hand TE something 'finished' for possible OS inclusion that they liked, they'd probably be way more excited to check that out than read all our feature requests ;)
  • At this point, it would make more sense to earn some other currency instead ;o))
  • Hahaha true :/
  • edited June 2016
    I've no idea on details/complications around redistribution of intellectual property law etc but (unless I'm missing some huge downside?) if I was TE I'd dig the principle of people messing with what's already there/adding to/trading online etc, so long as everything stayed 'op-1' and no one was hijacking part/all of TE's work as a basis or element in making new hardware/software for profit etc. I'd imagine it being a 'good' thing for Op sales if people were publicly sharing mods etc? Might be worth speaking to TE about it if it someone ends up making a sweet custom OS/additions etc? See how they feel about the idea..
  • edited June 2016

    I've no idea on the details/complications around laws concerning the redistribution of intellectual property etc but (unless I'm missing some huge downside?) if I was TE I'd dig the principle of people messing with what's already there/adding to/trading online etc so long as everything stayed 'op-1' and no one was hijacking part/all of their work as a basis or element in making new hardware/software for profit etc. I'd imagine it being a 'good' thing for Op sales if people were publicly sharing mods etc? Might be worth speaking to TE about it if it turns out that anyone digs deep enough for it to be potentially worthwhile sharing a custom OS.. See how they feel about it...

    Like some sort of a JJOS situation with Akai. Although JJ took the opportunity to develop the MPC 1/2.5k OS where Akai clearly gave up ages ago. Pretty sure MPC sales did benefit from it, so did JJ and the users obviously.

  • edited June 2016
    @lefilou yeah that kind of situation except I guess it would need to be more of a 'community' vibe as I'm pretty sure TE wouldn't be down with people selling stuff whilst TE are still developing/selling OP-1...so there'd have to be some pretty generous dudes out there with a labour of love for the Op OS...

    Edit - feel like I've slightly derailed the thread, sorry, I'll bail and leave you tech guys to get on with it ;)
  • Just so nobody gets their hopes up in this thread: I will not post any complete firmware here as that would contain Teenage Engineerings intellectual property and I would be redistributing it, which is copyright infringement (see cyanogen mod for android).

    That's understood from the get go, @TabascoEye. There's a lot of necessary research missing anyway, and any custom firmware would of course need its own graphics and code. As I mentioned, there are multiple stops like SDK cost, copyright, or lack of documentation, but I know the DIY community never gives up on researching, this is good stuff you're posting.
     

  • edited June 2016
    @TabascoEye

    Any idea why the unpacked tar has weird permissions set on it (on unix)? I can't acces them on the same user that extracted the files (permission denied), but as root the permissions look totally normal. Just realized they are stored as a different user in the original tar.

    Also did you need to use some special way to repackage the tar? I'm not getting the same file size when comparing my tar and the original tar :/ Edit: got the same size with no compression, but the contents look totally different in a hex editor :( I see that at least the file owner and/or permissions are different ofcourse in the new tar. It works on my desktop fine though. Wondering if it'll work in the OP...

Sign In or Register to comment.