synth4
a text based audio-visual modular-synthesizer
- the scripts it uses can be edited and debugged in realtime
- three parts to it, audio generation, graphics generation, and a fullscreen post-effects chain
- all variables used in either the audio or graphics code can be exported to the other side. this allows using things like audio oscillators and filter positions to modulate positions of graphics primitives, and using graphics primitive positions to modulate the audio
- audio can be routed to the post-effects shaders
- graphics primitives are currently all signed distance fields that can be chained together
- very basic support for midi 1.0 devices
- currently pre-alpha, but fairly stable. it has only been tested so far on a 2020 M1 macbook air.
download v0.7.3.f75b7d (requires osx 14.4) (2/7/2024)
simple video demonstration
usage
the program is essentially a text editor with some additional modes
- command+shift+i = variable-viewer mode. as the cursor moves around, any symbols underneath it have their current contents displayed. this is primarily useful for debugging the output of functions throughout the script. when viewing float_arrays, tab can be pressed to alternate between a 'current value' display, and a 'history' display.
- command+shift+m = midi mode. this lets the computer keyboard be used to send midi inputs into the script. the mapping from keys to midi note indices can be seen and edited in the settings.txt file
- command+shift+k = console mode. a list of commands can be seen by pressing tab while in this mode. tab will also autocomplete the commands where possible.
- use escape to back out of these modes
- pressing escape in the basic edit mode will show/hide the script
keyboard shortcuts for text navigation
- arrows to move the cursor
- arrows+command to move the cursor to the start and end of line or file
- arrows+option to move the cursor word by word
- all of the above + shift will generate text selections
- command+c/command+v for copy/paste (this uses the os, so copy/paste to/from other programs should work)
- command+z/command+z+shift for undo/redo
the following txt file is an annotated script file that explains the basics. the program supports drag and drop, so just drag the txt file into the window and it should start playing
example_scene.txt
unordered roadmap
- windows port
- general
- performance improvements
- replays/recording
- autosave
- more extensive midi support
- script editing
- stopping dropouts when script lines contain errors
- improved error messages in script
- autocomplete
- documentation in the program itself
- audio
- delays, reverb, echo, etc
- panning, equalisers, mixers, compressors, etc
- 3d sound using inputs from the visuals thread
- etc
- graphics
- lighting, materials, textures, etc
- volumetrics probably
- particle systems
- generative meshes
- video streaming?
- multiple scenes that can be masked and mixed together in the post effects chain?
- 2d distance fields
- etc, etc
script functions
INTEGER operator + (INTEGER a, INTEGER b)
FLOAT operator + (FLOAT a, FLOAT b)
FLOAT operator + (FLOAT a, INTEGER b)
FLOAT operator + (INTEGER a, FLOAT b)
BUFFER operator + (BUFFER a, BUFFER b)
BUFFER operator + (NUMBER a, BUFFER b)
BUFFER operator + (BUFFER a, NUMBER b)
SHAPE operator + (SHAPE shape a, SHAPE shape b)
FLOAT operator + (FLOAT a, FLOAT b)
FLOAT2 operator + (FLOAT2 a, FLOAT2 b)
FLOAT3 operator + (FLOAT3 a, FLOAT3 b)
FLOAT4 operator + (FLOAT4 a, FLOAT4 b)
FLOAT2 operator + (FLOAT a, FLOAT2 b)
FLOAT3 operator + (FLOAT a, FLOAT3 b)
FLOAT4 operator + (FLOAT a, FLOAT4 b)
FLOAT operator + (INTEGER a, FLOAT b)
FLOAT2 operator + (INTEGER a, FLOAT2 b)
FLOAT3 operator + (INTEGER a, FLOAT3 b)
FLOAT4 operator + (INTEGER a, FLOAT4 b)
FLOAT2 operator + (FLOAT2 a, FLOAT b)
FLOAT3 operator + (FLOAT3 a, FLOAT b)
FLOAT4 operator + (FLOAT4 a, FLOAT b)
FLOAT operator + (FLOAT a, INTEGER b)
FLOAT2 operator + (FLOAT2 a, INTEGER b)
FLOAT3 operator + (FLOAT3 a, INTEGER b)
FLOAT4 operator + (FLOAT4 a, INTEGER b)
INTEGER operator + (INTEGER a, INTEGER b)
INTEGER operator - (INTEGER a, INTEGER b)
FLOAT operator - (FLOAT a, FLOAT b)
FLOAT operator - (FLOAT a, INTEGER b)
FLOAT operator - (INTEGER a, FLOAT b)
BUFFER operator - (BUFFER a, BUFFER b)
BUFFER operator - (NUMBER a, BUFFER b)
BUFFER operator - (BUFFER a, NUMBER b)
INTEGER operator - (INTEGER a)
FLOAT operator - (FLOAT a)
BUFFER operator - (BUFFER a)
SHAPE operator - (SHAPE shape a, SHAPE shape b)
FLOAT operator - (FLOAT a, FLOAT b)
FLOAT2 operator - (FLOAT2 a, FLOAT2 b)
FLOAT3 operator - (FLOAT3 a, FLOAT3 b)
FLOAT4 operator - (FLOAT4 a, FLOAT4 b)
FLOAT2 operator - (FLOAT a, FLOAT2 b)
FLOAT3 operator - (FLOAT a, FLOAT3 b)
FLOAT4 operator - (FLOAT a, FLOAT4 b)
FLOAT operator - (INTEGER a, FLOAT b)
FLOAT2 operator - (INTEGER a, FLOAT2 b)
FLOAT3 operator - (INTEGER a, FLOAT3 b)
FLOAT4 operator - (INTEGER a, FLOAT4 b)
FLOAT2 operator - (FLOAT2 a, FLOAT b)
FLOAT3 operator - (FLOAT3 a, FLOAT b)
FLOAT4 operator - (FLOAT4 a, FLOAT b)
FLOAT operator - (FLOAT a, INTEGER b)
FLOAT2 operator - (FLOAT2 a, INTEGER b)
FLOAT3 operator - (FLOAT3 a, INTEGER b)
FLOAT4 operator - (FLOAT4 a, INTEGER b)
INTEGER operator - (INTEGER a, INTEGER b)
FLOAT operator - (FLOAT a)
FLOAT2 operator - (FLOAT2 a)
FLOAT3 operator - (FLOAT3 a)
FLOAT4 operator - (FLOAT4 a)
INTEGER operator / (INTEGER a, INTEGER b)
FLOAT operator / (FLOAT a, FLOAT b)
FLOAT operator / (FLOAT a, INTEGER b)
FLOAT operator / (INTEGER a, FLOAT b)
BUFFER operator / (BUFFER a, BUFFER b)
BUFFER operator / (NUMBER a, BUFFER b)
BUFFER operator / (BUFFER a, NUMBER b)
FLOAT operator / (FLOAT a, FLOAT b)
FLOAT2 operator / (FLOAT2 a, FLOAT2 b)
FLOAT3 operator / (FLOAT3 a, FLOAT3 b)
FLOAT4 operator / (FLOAT4 a, FLOAT4 b)
FLOAT2 operator / (FLOAT a, FLOAT2 b)
FLOAT3 operator / (FLOAT a, FLOAT3 b)
FLOAT4 operator / (FLOAT a, FLOAT4 b)
FLOAT operator / (INTEGER a, FLOAT b)
FLOAT2 operator / (INTEGER a, FLOAT2 b)
FLOAT3 operator / (INTEGER a, FLOAT3 b)
FLOAT4 operator / (INTEGER a, FLOAT4 b)
FLOAT2 operator / (FLOAT2 a, FLOAT b)
FLOAT3 operator / (FLOAT3 a, FLOAT b)
FLOAT4 operator / (FLOAT4 a, FLOAT b)
FLOAT operator / (FLOAT a, INTEGER b)
FLOAT2 operator / (FLOAT2 a, INTEGER b)
FLOAT3 operator / (FLOAT3 a, INTEGER b)
FLOAT4 operator / (FLOAT4 a, INTEGER b)
INTEGER operator / (INTEGER a, INTEGER b)
INTEGER operator * (INTEGER a, INTEGER b)
FLOAT operator * (FLOAT a, FLOAT b)
FLOAT operator * (FLOAT a, INTEGER b)
FLOAT operator * (INTEGER a, FLOAT b)
BUFFER operator * (BUFFER a, BUFFER b)
BUFFER operator * (NUMBER a, BUFFER b)
BUFFER operator * (BUFFER a, NUMBER b)
SHAPE operator * (SHAPE shape, TRANSFORM transform)
SHAPE operator * (TRANSFORM transform, SHAPE shape)
TRANSFORM operator * (TRANSFORM transform_a, TRANSFORM transform_b)
FLOAT operator * (FLOAT a, FLOAT b)
FLOAT2 operator * (FLOAT2 a, FLOAT2 b)
FLOAT3 operator * (FLOAT3 a, FLOAT3 b)
FLOAT4 operator * (FLOAT4 a, FLOAT4 b)
FLOAT2 operator * (FLOAT a, FLOAT2 b)
FLOAT3 operator * (FLOAT a, FLOAT3 b)
FLOAT4 operator * (FLOAT a, FLOAT4 b)
FLOAT operator * (INTEGER a, FLOAT b)
FLOAT2 operator * (INTEGER a, FLOAT2 b)
FLOAT3 operator * (INTEGER a, FLOAT3 b)
FLOAT4 operator * (INTEGER a, FLOAT4 b)
FLOAT2 operator * (FLOAT2 a, FLOAT b)
FLOAT3 operator * (FLOAT3 a, FLOAT b)
FLOAT4 operator * (FLOAT4 a, FLOAT b)
FLOAT operator * (FLOAT a, INTEGER b)
FLOAT2 operator * (FLOAT2 a, INTEGER b)
FLOAT3 operator * (FLOAT3 a, INTEGER b)
FLOAT4 operator * (FLOAT4 a, INTEGER b)
INTEGER operator * (INTEGER a, INTEGER b)
INTEGER operator == (NUMBER a, NUMBER b)
INTEGER operator == (NUMBER a, NUMBER b)
INTEGER operator != (NUMBER a, NUMBER b)
INTEGER operator != (NUMBER a, NUMBER b)
INTEGER operator > (NUMBER a, NUMBER b)
INTEGER operator > (NUMBER a, NUMBER b)
INTEGER operator >= (NUMBER a, NUMBER b)
INTEGER operator >= (NUMBER a, NUMBER b)
INTEGER operator < (NUMBER a, NUMBER b)
INTEGER operator < (NUMBER a, NUMBER b)
INTEGER operator <= (NUMBER a, NUMBER b)
INTEGER operator <= (NUMBER a, NUMBER b)
INTEGER operator and (INTEGER a, INTEGER b)
INTEGER operator and (INTEGER a, INTEGER b)
INTEGER operator or (INTEGER a, INTEGER b)
INTEGER operator or (INTEGER a, INTEGER b)
INTEGER operator not (INTEGER a)
INTEGER max(INTEGER a, INTEGER b)
FLOAT max(FLOAT a, FLOAT b)
FLOAT max(FLOAT a, INTEGER b)
FLOAT max(INTEGER a, FLOAT b)
BUFFER max(BUFFER a, BUFFER b)
BUFFER max(NUMBER a, BUFFER b)
BUFFER max(BUFFER a, NUMBER b)
FLOAT max(FLOAT a, FLOAT b)
FLOAT2 max(FLOAT2 a, FLOAT2 b)
FLOAT3 max(FLOAT3 a, FLOAT3 b)
FLOAT4 max(FLOAT4 a, FLOAT4 b)
INTEGER min(INTEGER a, INTEGER b)
FLOAT min(FLOAT a, FLOAT b)
FLOAT min(FLOAT a, INTEGER b)
FLOAT min(INTEGER a, FLOAT b)
BUFFER min(BUFFER a, BUFFER b)
BUFFER min(NUMBER a, BUFFER b)
BUFFER min(BUFFER a, NUMBER b)
FLOAT min(FLOAT a, FLOAT b)
FLOAT2 min(FLOAT2 a, FLOAT2 b)
FLOAT3 min(FLOAT3 a, FLOAT3 b)
FLOAT4 min(FLOAT4 a, FLOAT4 b)
INTEGER atan2(INTEGER a, INTEGER b)
FLOAT atan2(FLOAT a, FLOAT b)
FLOAT atan2(FLOAT a, INTEGER b)
FLOAT atan2(INTEGER a, FLOAT b)
BUFFER atan2(BUFFER a, BUFFER b)
BUFFER atan2(NUMBER a, BUFFER b)
BUFFER atan2(BUFFER a, NUMBER b)
FLOAT atan2(FLOAT a, FLOAT b)
FLOAT2 atan2(FLOAT2 a, FLOAT2 b)
FLOAT3 atan2(FLOAT3 a, FLOAT3 b)
FLOAT4 atan2(FLOAT4 a, FLOAT4 b)
INTEGER pow(INTEGER a, INTEGER b)
FLOAT pow(FLOAT a, FLOAT b)
FLOAT pow(FLOAT a, INTEGER b)
FLOAT pow(INTEGER a, FLOAT b)
BUFFER pow(BUFFER a, BUFFER b)
BUFFER pow(NUMBER a, BUFFER b)
BUFFER pow(BUFFER a, NUMBER b)
FLOAT pow(FLOAT a, FLOAT b)
FLOAT2 pow(FLOAT2 a, FLOAT2 b)
FLOAT3 pow(FLOAT3 a, FLOAT3 b)
FLOAT4 pow(FLOAT4 a, FLOAT4 b)
INTEGER abs(INTEGER a)
FLOAT abs(FLOAT a)
BUFFER abs(BUFFER a)
FLOAT abs(FLOAT a)
FLOAT2 abs(FLOAT2 a)
FLOAT3 abs(FLOAT3 a)
FLOAT4 abs(FLOAT4 a)
INTEGER sin(INTEGER a)
FLOAT sin(FLOAT a)
BUFFER sin(BUFFER a)
SHAPE sin(SHAPE shape, NUMBER frequency, NUMBER amplitude, NUMBER phase_offset)
FLOAT sin(FLOAT a)
FLOAT2 sin(FLOAT2 a)
FLOAT3 sin(FLOAT3 a)
FLOAT4 sin(FLOAT4 a)
INTEGER cos(INTEGER a)
FLOAT cos(FLOAT a)
BUFFER cos(BUFFER a)
FLOAT cos(FLOAT a)
FLOAT2 cos(FLOAT2 a)
FLOAT3 cos(FLOAT3 a)
FLOAT4 cos(FLOAT4 a)
INTEGER tan(INTEGER a)
FLOAT tan(FLOAT a)
BUFFER tan(BUFFER a)
FLOAT tan(FLOAT a)
FLOAT2 tan(FLOAT2 a)
FLOAT3 tan(FLOAT3 a)
FLOAT4 tan(FLOAT4 a)
INTEGER asin(INTEGER a)
FLOAT asin(FLOAT a)
BUFFER asin(BUFFER a)
FLOAT asin(FLOAT a)
FLOAT2 asin(FLOAT2 a)
FLOAT3 asin(FLOAT3 a)
FLOAT4 asin(FLOAT4 a)
INTEGER acos(INTEGER a)
FLOAT acos(FLOAT a)
BUFFER acos(BUFFER a)
FLOAT acos(FLOAT a)
FLOAT2 acos(FLOAT2 a)
FLOAT3 acos(FLOAT3 a)
FLOAT4 acos(FLOAT4 a)
INTEGER atan(INTEGER a)
FLOAT atan(FLOAT a)
BUFFER atan(BUFFER a)
FLOAT atan(FLOAT a)
FLOAT2 atan(FLOAT2 a)
FLOAT3 atan(FLOAT3 a)
FLOAT4 atan(FLOAT4 a)
INTEGER exp(INTEGER a)
FLOAT exp(FLOAT a)
BUFFER exp(BUFFER a)
FLOAT exp(FLOAT a)
FLOAT2 exp(FLOAT2 a)
FLOAT3 exp(FLOAT3 a)
FLOAT4 exp(FLOAT4 a)
INTEGER exp2(INTEGER a)
FLOAT exp2(FLOAT a)
BUFFER exp2(BUFFER a)
FLOAT exp2(FLOAT a)
FLOAT2 exp2(FLOAT2 a)
FLOAT3 exp2(FLOAT3 a)
FLOAT4 exp2(FLOAT4 a)
INTEGER expm1(INTEGER a)
FLOAT expm1(FLOAT a)
BUFFER expm1(BUFFER a)
FLOAT expm1(FLOAT a)
FLOAT2 expm1(FLOAT2 a)
FLOAT3 expm1(FLOAT3 a)
FLOAT4 expm1(FLOAT4 a)
INTEGER log(INTEGER a)
FLOAT log(FLOAT a)
BUFFER log(BUFFER a)
FLOAT log(FLOAT a)
FLOAT2 log(FLOAT2 a)
FLOAT3 log(FLOAT3 a)
FLOAT4 log(FLOAT4 a)
INTEGER log10(INTEGER a)
FLOAT log10(FLOAT a)
BUFFER log10(BUFFER a)
FLOAT log10(FLOAT a)
FLOAT2 log10(FLOAT2 a)
FLOAT3 log10(FLOAT3 a)
FLOAT4 log10(FLOAT4 a)
INTEGER log2(INTEGER a)
FLOAT log2(FLOAT a)
BUFFER log2(BUFFER a)
FLOAT log2(FLOAT a)
FLOAT2 log2(FLOAT2 a)
FLOAT3 log2(FLOAT3 a)
FLOAT4 log2(FLOAT4 a)
INTEGER log1p(INTEGER a)
FLOAT log1p(FLOAT a)
BUFFER log1p(BUFFER a)
FLOAT log1p(FLOAT a)
FLOAT2 log1p(FLOAT2 a)
FLOAT3 log1p(FLOAT3 a)
FLOAT4 log1p(FLOAT4 a)
INTEGER sqrt(INTEGER a)
FLOAT sqrt(FLOAT a)
BUFFER sqrt(BUFFER a)
FLOAT sqrt(FLOAT a)
FLOAT2 sqrt(FLOAT2 a)
FLOAT3 sqrt(FLOAT3 a)
FLOAT4 sqrt(FLOAT4 a)
FLOAT acc(NUMBER increment)
FLOAT acc(NUMBER increment, NUMBER offset)
BUFFER history(BUFFER buffer)
FLOAT mix(FLOAT a, FLOAT b, FLOAT blend)
INTEGER mix(INTEGER a, INTEGER b, FLOAT blend)
BUFFER mix(BUFFER a, BUFFER b, FLOAT blend)
BUFFER mix(BUFFER a, BUFFER b, BUFFER blend)
SHAPE mix(SHAPE shape a, SHAPE shape b, FLOAT blend)
FLOAT4 sample(TEXTURE texture, FLOAT2 uv)
FLOAT4 sample(TEXTURE texture, FLOAT2 uv, STRING options)
FLOAT4 sample(TEXTURE texture, FLOAT u, FLOAT v)
FLOAT4 sample(TEXTURE texture, FLOAT u, FLOAT v, STRING options)
FLOAT sample(BUFFER audio, FLOAT u)
FLOAT sum(FLOAT values)
INTEGER sum(INTEGER values)
BUFFER sum(BUFFER values)
ANYTHING import(STRING name)
VOID export(FLOAT value)
VOID export(STRING name, FLOAT value)
VOID export(BUFFER value)
VOID export(STRING name, BUFFER value)
VOID export(INTEGER value)
VOID export(STRING name, INTEGER value)
BUFFER adsr(NUMBER|BUFFER attack time, NUMBER|BUFFER decay time, NUMBER|BUFFER sustain level, NUMBER|BUFFER release time, BUFFER signals)
BUFFER osc(STRING wave_table, NUMBER|BUFFER frequency, NUMBER|BUFFER amplitude)
BUFFER osc(BUFFER wave_table, NUMBER|BUFFER frequency, NUMBER|BUFFER amplitude)
FLOAT lfo(STRING lfo_type, NUMBER frequency)
FLOAT lfo(STRING lfo_type, NUMBER frequency, NUMBER amplitude)
FLOAT lfo(STRING lfo_type, NUMBER frequency, NUMBER amplitude, NUMBER phase_offset)
BUFFER lfo_a(STRING lfo_type, NUMBER frequency, NUMBER amplitude)
BUFFER lfo_a(STRING lfo_type, NUMBER frequency, NUMBER amplitude, NUMBER phase_offset)
STRING euc(NUMBER k, NUMBER n)
STRING euc(NUMBER k, NUMBER n, NUMBER shift)
BUFFER seq(NUMBER bpm, STRING pattern)
BUFFER lowpass(BUFFER signal, NUMBER cutoff frequency)
BUFFER highpass(BUFFER signal, NUMBER cutoff frequency)
BUFFER bandpass(BUFFER signal, NUMBER min cutoff frequency, NUMBER max cutoff frequency)
BUFFER bandstop(BUFFER signal, NUMBER min cutoff frequency, NUMBER max cutoff frequency)
FLOAT rng()
FLOAT rng(FLOAT min, FLOAT exclusive_max)
BUFFER rng(BUFFER min, BUFFER exclusive_max)
BUFFER white_noise()
INTEGER notes(STRING notes)
INTEGER notes(STRING scale, STRING root_note)
BUFFER note_signals(INTEGER notes)
BUFFER note_signals(INTEGER notes, STRING device)
FLOAT note_hz(INTEGER notes)
FLOAT note_hz(INTEGER notes, STRING device)
FLOAT4 colour(FLOAT r)
FLOAT4 colour(FLOAT r, FLOAT g, FLOAT b)
FLOAT4 colour(FLOAT r, FLOAT g, FLOAT b, FLOAT a)
FLOAT4 colour(FLOAT3 rgb, FLOAT a)
SHAPE cone()
SHAPE cone(NUMBER radius, NUMBER height)
SHAPE cube()
SHAPE cube(NUMBER width, NUMBER height, NUMBER depth)
SHAPE cube(NUMBER size)
SHAPE cylinder()
SHAPE cylinder(NUMBER radius, NUMBER height)
SHAPE pyramid()
SHAPE pyramid(NUMBER radius, NUMBER height)
SHAPE torus()
SHAPE torus(NUMBER large_radius, NUMBER small_radius)
SHAPE sphere()
SHAPE sphere(NUMBER radius)
SHAPE union(SHAPE shape a, SHAPE shape b)
SHAPE union(SHAPE shape a, SHAPE shape b, NUMBER blend)
SHAPE difference(SHAPE shape a, SHAPE shape b)
SHAPE difference(SHAPE shape a, SHAPE shape b, NUMBER blend)
SHAPE intersection(SHAPE shape a, SHAPE shape b)
SHAPE intersection(SHAPE shape a, SHAPE shape b, NUMBER blend)
SHAPE reflect(SHAPE shape)
SHAPE twist(SHAPE shape, NUMBER rate)
TRANSFORM matrix()
TRANSFORM rotate_x(TRANSFORM transform, NUMBER angle)
TRANSFORM rotate_x(NUMBER angle)
TRANSFORM rotate_y(TRANSFORM transform, NUMBER angle)
TRANSFORM rotate_y(NUMBER angle)
TRANSFORM rotate_z(TRANSFORM transform, NUMBER angle)
TRANSFORM rotate_z(NUMBER angle)
TRANSFORM translate_x(TRANSFORM transform, NUMBER x)
TRANSFORM translate_x(NUMBER x)
TRANSFORM translate_y(TRANSFORM transform, NUMBER y)
TRANSFORM translate_y(NUMBER y)
TRANSFORM translate_z(TRANSFORM transform, NUMBER z)
TRANSFORM translate_z(NUMBER z)
TRANSFORM translate(TRANSFORM transform, NUMBER x, NUMBER y, NUMBER z)
TRANSFORM translate(NUMBER x, NUMBER y, NUMBER z)
TRANSFORM inverse(TRANSFORM transform)
VOID draw(SHAPE shape)
VOID draw(SHAPE shape, TRANSFORM transform)
change log
last updated 27/6/2024