Developer Interview: Robin Lobel

Can you tell us a little bit about yourself (who are you, what projects have you been involved with in the past, what drives and motivates you, etc.)?

I live in Paris and I’ve been researching and developing technologies for the broadcast and media industries for years. My work has ranged from 3D to video to audio algorithms and software solutions. I’m mostly self-taught, with a focus on the links between the real and digital worlds – making real world data easy to edit.

After working on my own research for a couple post-production companies, in 2009 I started my own company, Divide Frame, by developing a GPU Decoder plugin for Vegas and Premiere. At that time, decoding h264 files from camcorders and cameras was painfully slow or required transcoding. The plugin I developed used the power of graphics card to decode h264 files on the fly with no transcoding required. Now, most NLEs do that.

Can you describe your vision for SpectraLayers Pro?

I’m not a big fan of one-click, black-box solutions and at that time, audio tools were mostly that – a couple knobs and sliders, a click and if you’re not fully satisfied with the end result, there’s no much else you can do.
It’s fine when your goal is straightforward: apply a special effect, do some EQ, silence or recut a part – classic editing. But as soon as you want – or need – to go beyond that, you’re stuck with a waveform shape that means nothing but power at certain time or in the best case scenario, a spectrogram with not much control over it. You can hear what’s wrong, sometimes you can visualize it, but you can only push a couple buttons and that’s it. What you really need is to get your hands directly on the spectral data in order to tweak it. Your brain doesn’t receive and process a waveform, it receives and processes a stream of frequencies. So, it makes much more sense to manipulate frequencies over time instead of a raw waveform shape. Hence, the birth of the spectrum-oriented approach in SpectraLayers.
A spectrum is like a picture made of sounds but I didn’t want to provide just a basic picture editor experience. You can’t do much image editing to a picture in Paint, can you ? What I needed to create had to be the full Photoshop-like package, with fully customizable tools, and layers for compositing.

Was there strict plan for the development? Or was it mainly just trial and error?

The very first version was a bit clunky – it was my first big software project and in the beginning it wasn’t even designed to be a a full spectral editor like what it is now. Halfway through the development, I felt I was doing it wrong and decided to restart everything from scratch. The foundations of SpectraLayers was set. Version two were improvements to version one and by the time I completed version two, I gained enough experience and decided to restart everything from scratch… once again!

Version three had a brand new engine, was much more optimized and provided a better user experience matching real picture editors to DAW workflows. It also had a better emphasis on layers with each layer now having its own distinct color. It was basically like a brand new software and it finally felt right.

What feature of SpectraLayers Pro provided the biggest development challenge and why?

On the engine side, the biggest challenge was designing a spectral engine that was highly accurate while remaining completely in realtime – no matter the length of the files, the settings or the number of layers or channels. Having worked with realtime 3D renderers and algorithms in the past, I used techniques from the video game industry to provide a similar experience to the user by decoupling the visual rendering to the editing and using every bit of hardware acceleration a computer can provide.

SpectraLayers Pro’s approach is really unique and outstanding on the market. What do you believe makes this software so special and distinct?

SpectraLayers’s engine is built upon three main principles:

What is the main focus for the new version?

Inter-operability, project and layer management and feature and engine improvements.

There was high demand for SpectraLayers to integrate with various DAWs, in particular with Avid Pro Tools. The new version now offers a Pro Tools plugin (AAX) which allows seamless inter-operability with your projects in Pro Tools. One-click transfer of audio data to SpectraLayers and one-click import of the modified data from SpectraLayers back to your Pro Tools project. You can view the program’s documentation for further integration with other DAWs.

Project management allows you to work on several projects in the same session (each having its own tab) and to easily transfer data and layers from one project to another, make backup copies of your current project, etc. With layer management, layers can now be grouped which is not only a management feature but also a mixing feature. Each new group acts as a virtual layer that shows the spectral and waveform results of mixing grouped layers while keeping each layer’s data and properties. This can be handy for non-destructive processing, like when you phase-invert specific spectral data into another layer to remove it (instead of destructively removing it from the original layer), or just to check spectral and waveform levels and clipping.

Finally, a couple features were improved, like the frequency and harmonic selection tools which now have a new option that tracks the frequency as you swipe the tool across the screen – making selections much easier and faster. The new version also uses a brand new high-quality realtime resampler engine to ensure the best audio playback and recording quality despite your hardware’s capabilities. Additionally, the history system now tracks removed, added and merged groups and layers, making easier to undo and redo your actions.

This is the most polished release to date and improves on the foundations set by SpectraLayers Pro 3.

 

SpectraLayers Pro is often used for sound design, forensics, audio cleaning and many other audio editing demands. What aspect of audio editing do you feel has the biggest potential?

While audio cleaning, forensics and extraction comes to mind with spectral data editing, there’s a whole new world to explore in audio mastering and sound design. So far, mastering is mainly done by adjusting volume envelopes and a couple of EQs. But what if you do that while watching the tracks in their 3D spectral representation? You could see overlapping frequencies and spectral powers between tracks and have direct control on spectral peaks and spectral areas that lack presence.

The same goes for special sound effects and sound design which are usually done by applying black-box generators and filters. What if you could draw spectral shapes directly and tweak their spectral appearance, apply local blur or twist individual frequencies? That would bring rise to a whole new kind of control and creative ability – a brand new audio world to explore.