Music Synth

Device Type
Handheld arpeggio synthesizer. A pocket instrument that loops one of four synthesized arpeggios through its own speaker; the musical mood, volume and transport live on the touch screen while the pitch lives on a physical rotary knob. It solves the 'I want a small, immediate, tunable arpeggio box' problem — no phone, no network, no setup: power on, press Play, turn the knob.

Hardware
chip: esp32s3 — board: m5-cores3 — boardName: M5Stack CoreS3 — gpios: consumedByProduct: GPIO 1 (SCL) and GPIO 2 (SDA) as the Grove-A I2C bus the knob unit sits on — declared by the product as a board_amend overlay so the board manager owns them — freeAfterWiring: 5 · 6 · 7 · 8 · 9 · 10 · 17 · 18 — reservedUntouched: 43 (console UART) — peripherals: name: M5Stack Unit Encoder — role: physical pitch knob — bus: I2C, Grove-A port (SDA/SCL), 3V3 + GND — i2cAddress: 0x40 (fixed default) — notes: Not a bare encoder: an STM32F030 unit that integrates a 30-pulse rotary knob (with push-button) and 2x SK6812 programmable RGB LEDs, all reached over I2C. Register map per M5Stack's own Unit_Encoder library: 0x00 work mode, 0x10 encoder value (signed 16-bit, little-endian, a RUNNING COUNT), 0x20 button status (0 = pressed, 1 = released), 0x30 LED write of [index, R, G, B] where index 0 addresses both LEDs, 0xFE firmware version. It never drives the port's two signal pins as GPIO — which is why an earlier revision that decoded them with the pulse counter could not see a turn at all. PORT.A POWER IS THE PREREQUISITE, and it is not automatic: the CoreS3 does NOT hard-wire 3V3 to Grove-A. The port is fed through the SY7088 boost converter and gated by the AW9523, and M5Stack's own M5Unified enables BOTH AW9523 Port0 bit 1 (BUS_EN, reg 0x02) and Port1 bit 7 (SY7088 BOOST_EN, reg 0x03). This product's board definition sets neither and actively drives Port0 bit 1 LOW at init, so the port came up dead and this unit could never answer — proven without the firmware by a Unit Step16 in PORT.A keeping its own 7-segment digit DARK (that digit is driven by the unit's own MCU, so dark = no 3V3). app_sound_enable_port_a_power() now sets both bits additively, and app_main calls it between esp_board_manager_init() and any task that talks to the port. — unusedCapability: None left unused: the knob drives the pitch and its RGB LED is the pitch meter, and the push-button is mapped to play/pause (f12). · name: CoreS3 onboard display — role: home screen — part: ili9341 320x240 LCD + ft5x06 capacitive touch — notes: Already on the board — nothing is wired for it. · name: CoreS3 onboard audio output — role: speaker — part: AW88298 I2S amplifier + speaker — notes: Already on the board — the synth feeds it over I2S; nothing external is added. The board definition gives this part no amplifier-enable GPIO and its power-control hook does not resolve, so the firmware switches the speaker path on itself (AXP2101 rails + the AW9523 expander's P0_2 speaker enable). · name: CoreS3 onboard power management — role: battery — part: AXP2101 + internal battery, USB-C charging · name: CoreS3 onboard microphone (ES7210) and camera (GC0308) — role: unused — notes: Present on the board but not used by this product. — synthesis: Generated on-chip by the ESP32-S3 — no samples, no audio files. One monophonic voice, 16-bit, mono, streamed over I2S to the AW88298 amplifier. Each mood has its own waveform, envelope and note pattern. SAMPLE RATE — SHIPS AT 16 kHz, A DELIBERATE AND OWNER-APPROVED DEVIATION FROM 44.1 kHz. The spec asks for 44.1 kHz, the codec accepts it, and its clock is genuinely accurate on this chip — but the speaker is SILENT at that rate and AUDIBLE at 16 kHz. Established by a controlled A/B on one board with only the rate changed (`synth rate 16000` / `synth rate 44100`, an in-place codec reopen, no reflash), reproduced in both directions and done with the owner listening. The owner chose to ship the rate already proven audible rather than spend a bench round testing a second candidate. WHAT IS AND IS NOT KNOWN ABOUT THE CAUSE. Two explanations were previously recorded here and BOTH were wrong, so they are corrected rather than quietly dropped. (1) It is NOT an integer-divider truncation: the I2S MCLK divider on this chip is FRACTIONAL (I2S_MCLK = SRC / (N + b/a), a,b up to 512), so 44.1 kHz's 11,289,600 Hz is produced exactly, as 160 MHz / (14 + 76/441). The bench log proves the fractional divider is active: at 16 kHz the driver reported MCLK 4,096,000 Hz, exactly 16000 x 256, which integer division could never produce (160e6/39 = 4,102,564). (2) The ITU "MCLK jitter ratio" line in the log (13% at 16 kHz, 34% at 44.1 kHz) is the driver's own heuristic computed from REQUESTED values, not a measurement of the hardware, and must not be used to predict audibility. A PLAUSIBLE MECHANISM, explicitly a hypothesis and not an established cause: a fractional divider reaches its average frequency by dithering its instantaneous integer value, so the MCLK period alternates. At 16 kHz the fraction is 1/16, so the divider alternates only occasionally; at 44.1 kHz it is 76/441 (an average divider of 5/6), so the period swings between 14 and 15 source clocks — roughly +-3.4% — and the AW88298's internal PLL may fail to lock on that, outputting nothing while every readable register still reports enabled, unmuted and at volume. This is unverified and is recorded as a hypothesis so it is not mistaken for fact a third time. It also explains why the failure is specific to this rate on this amplifier, and why nothing the firmware can read reveals it: the amplifier's lock state is not exposed in any register. HOW THE FIRMWARE HANDLES IT. The product boots at 16 kHz, and the rate is still chosen in code rather than hardcoded so that hardware which CAN run 44.1 kHz (an audio PLL, an exact external MCLK, a different amplifier) gets the spec's rate back without a source change. Each candidate rate's clock is programmed and its ACHIEVED MCLK read back from the peripheral (`synth mclk` reports this, along with the divider the hardware settled on); a rate whose clock cannot be produced is skipped. The boot order is: 16 kHz (the proven-audible rate), then 48 kHz as a safety fallback, then the spec's 44.1 kHz only if its clock checks out — and the final fallback is never the rate known to be silent, so the product cannot boot as a silent box. `synth rate <hz>` reopens the codec live to repeat the A/B. The mood timing is derived from the ACTIVE rate, so the fixed 120 BPM tempo is 120 BPM at every rate in the ladder rather than only at 44.1 kHz.

Inputs
kind: rotary-knob — label: Pitch knob — hardware: M5Stack Unit Encoder, I2C 0x40, encoder-value register 0x10 — behavior: Polled over I2C every 20 ms (the unit has no interrupt line). The unit reports a RUNNING COUNT rather than a position, so each read is turned into a delta and accumulated into the pitch — which wraps, so 47 goes back to 0 and 0 back to 47 rather than stopping. How far one turn moves the pitch is the KNOB SENSITIVITY, adjustable on the touch screen (see the Knob sensitivity input): 1/8x, 1/4x, 1x (default), 2x or 4x — that is 96, 48, 24, 12 or 6 of the unit's counts per pitch step, so a lower level means a longer turn and finer control. At the default 1x one turn covers roughly half the 110-880 Hz sweep. Turning the knob is the only way to change pitch, and it works whether the synth is playing or paused — the readout and the LED colour update live either way. Rotation direction follows the unit's own default (clockwise = up); invert ENCODER_COUNT_SCALE in app_config.h if it feels backwards. · kind: rotary-button — label: Knob button — hardware: the encoder unit's own push-button, I2C register 0x20 (0 = pressed) — behavior: A press toggles play/pause, exactly like the on-screen transport button, so the transport is reachable while both hands are on the knob. Detected on the press EDGE, so a stuck button cannot sit there toggling. · kind: touch-button — label: Play / Pause — hardware: ft5x06 touch on the onboard LCD — behavior: Tapping toggles between playing and paused. Pause silences the output mid-pattern and releases the voice; Play restarts the current mood's pattern from its first step. State is shown by the button's own label/icon. · kind: touch-slider — label: Volume — hardware: ft5x06 touch on the onboard LCD — behavior: Horizontal slider, 0-100%, boot value 50%. Drag to set; the value applies immediately and is smoothed over ~20 ms so there is no click. Mapped to the amplifier's digital gain. Volume is independent of play/pause — it can be moved while paused. · kind: touch-stepper — label: Knob sensitivity — hardware: ft5x06 touch on the onboard LCD — behavior: A − / + pair on the home screen stepping the pitch knob's sensitivity through five levels, shown as the multiplier the setting is reasoned in: 1/8x, 1/4x, 1x (default), 2x, 4x — i.e. 96, 48, 24, 12 or 6 of the unit's counts per pitch step, so a higher level means the pitch moves further per turn. Takes effect on the very next turn of the knob (the encoder picks the value up on its next 20 ms poll), and the partial step is carried across the change so adjusting it mid-turn neither skips nor loses pitch. The product stores nothing, so it returns to 1x on every boot. Also settable from the console as `synth sens <level|1x..4x|counts>`. · kind: touch-selector — label: Mood selection — hardware: ft5x06 touch on the onboard LCD — behavior: Four mood entries (Ambient, Classical, Chiptune, Acid) on the home screen. Tapping one switches the synth voice immediately; if the synth is playing, the new mood's arpeggio continues seamlessly from that point on — there is no need to press Stop first. The selected mood's name is displayed prominently on screen, and its entry is shown as highlighted.

Outputs
kind: speaker — label: Arpeggio audio — hardware: AW88298 I2S amplifier + onboard speaker — behavior: The product's whole reason for existing. While playing, loops the current mood's arpeggio in the current pitch and volume. While paused, output is silent — no idle hiss or leftover tone. The pattern is stepped by the fixed 120 BPM tempo, so each mood has a steady, metronomic pulse. · kind: display — label: Home screen — hardware: ili9341 320x240 LCD, portrait orientation — behavior: One screen, always visible while the device runs. Shows: the current mood's name (largest element), a play/pause button reflecting whether audio is running, a volume slider with its 0-100% value, the current pitch as a frequency readout in Hz, and the four mood entries. States: 'playing' (button shows pause, mood name normal), 'paused' (button shows play), 'knob missing' (a small note replaces the frequency readout). · kind: indicator — label: Pitch meter (knob unit's RGB LEDs) — hardware: the encoder unit's 2x SK6812 RGB LEDs, I2C register 0x30 written as [index, R, G, B] — behavior: Pitch meter. Colour is interpolated with the pitch: cool blue (0,0,255) at the lowest frequency, through cyan/green/yellow, to red (255,0,0) at the highest. Updated on every knob change at full brightness — not modulated by volume. It reads as 'how hot is this pitch'. The unit ships with its RGB enabled but black, so the firmware writes a colour at start-up or the LED stays dark.

Business Logic
powerOnBehavior: Power on (USB-C or battery) → the CoreS3's own boot, then PORT.A is powered (AW9523 BUS_EN + SY7088 BOOST_EN) and the synth comes up in the paused state and stays silent: mood Ambient, volume 50%, pitch at the middle of the sweep, output muted until Play is tapped. The audio engine opens at 16 kHz, the rate this board's amplifier is audible at (see the synthesis note on the rate deviation). Nothing is restored from a previous session — see factoryReset for why there is no stored state. If the knob unit is not answering, the device still starts as a working synth with the pitch at the middle of the range and a 'knob not found' note on screen; it keeps retrying so the unit can be plugged in later. — automationRules: No automations, schedules or scene integration — the product is entirely local and manual by design. The only implicit rule is that the device never produces sound without a deliberate Play tap. — dayToDayOperation: Turn it on: silent, Ambient shown, the tuning readout showing the middle of the range, the meter LED painted blue-to-red for it, and the audio engine open at 16 kHz — the rate this board's amplifier is audible at, so Play gives sound immediately with no console command needed. Tap Play: the Ambient arpeggio starts looping. Turn the knob: the pitch sweeps continuously across three octaves, applying on the next arpeggio step with a short ~20 ms glide so there is no zipper noise; the on-screen frequency readout and the RGB colour both follow immediately. If one turn moves the pitch too fast or too slow, tap − or + on the 'knob 1x' row to change the sensitivity and turn it again — the new feel applies on the very next turn, with no reboot or reflash. Press the knob's button to toggle play/pause without leaving the knob. Tap Classical/Chiptune/Acid: the voice changes instantly and the arpeggio carries on without a pause — this is how a player auditions the four moods. Drag the volume slider: level moves smoothly. Tap Pause: silence. Tap Play again: the pattern restarts from its first note. Turning the knob while paused is fine — it just re-aims the pitch for when Play is pressed. Note that pitch changes are NOT snapped to musical semitones: the knob is a continuous sweep across three octaves, in keeping with the requested 'sweep more than a melody' feel.

Factory Reset
trigger: Power off and on again (on/off via the CoreS3's side buttons, or unplug and replug USB-C while on battery). — effect: Full factory state, because the product deliberately stores nothing: mood returns to Ambient, volume to 50%, transport to paused and silent, and the pitch to the middle of the sweep. There is no user data, no Wi-Fi credentials and no saved settings to clear, so a simple power cycle is the complete reset — a stuck device is fixed by turning it off and on.

Failure Modes
scenario: Knob not connected, or PORT.A unpowered. — behavior: The synth still boots and plays — the knob is an accessory, not a dependency. Pitch falls back to the middle of the range, the screen shows a small 'knob not found' note in place of the frequency readout, and the firmware retries the I2C probe periodically (promptly for the first few seconds after boot, since the port's power comes up at boot, then backing off) so plugging the knob in later lights it up without a reboot. `synth audio` reports whether the port's BUS_EN and BOOST_EN bits are actually set, so an unpowered port is distinguishable from an absent unit without a spare part. · scenario: Knob unplugged while the synth is playing. — behavior: Audio continues uninterrupted at the last pitch, so a bumped cable never cuts the sound. Further LED writes simply fail and are ignored; the on-screen note reappears. · scenario: I2C bus disturbance or a mis-addressed knob unit (its address is fixed at 0x40 but another unit may be present). — behavior: Treated exactly as 'knob missing' — fails soft, never hangs the audio path. The firmware never blocks on an I2C read and sets a short (50 ms) timeout. · scenario: Power blip, brownout or the battery running flat mid-session. — behavior: The device reboots into the paused, silent state, and re-powers PORT.A. Because boot is always paused, a power event can never make the instrument suddenly start playing at full volume in a quiet room. If the battery is critically low the AXP2101's own shutdown takes over; nothing is lost because nothing was stored. · scenario: Volume set high with a bass-heavy mood (Acid) on the small speaker. — behavior: A soft limiter on the output stage prevents hard digital clipping, so the top of the volume range sounds loud and compressed rather than crackly. · scenario: Rapid knob spinning. — behavior: The unit counts pulses in its own microcontroller, so nothing is lost between polls; the driver takes the difference between consecutive reads and the audio glides to the new pitch on the next step instead of producing a burst of tones. · scenario: No network available. — behavior: Not a failure mode — the product is fully offline and has no connectivity to lose.

Power Management
batteryPowered: true — sleepMode: None. The audio engine stays live while playing and the screen stays on, since the instrument is meant to be picked up and played immediately. Battery life is preserved by powering the unit off when finished; there is no auto-sleep to surprise the player by muting mid-pattern.

Custom Behavior
Synth engine: single monophonic voice generated on the ESP32-S3 at 44.1 kHz / 16-bit mono, streamed to the onboard AW88298 amplifier over I2S. No stored samples and no files on flash. · Mood definitions — Ambient: slow evolving pad, long attack and release, soft attack waveform, wide-spaced notes, slowest feel. Classical: arpeggiated triad, clean plucked envelope with fast decay, notes rising through a major triad. Chiptune: 8-bit style square wave lead, hard edges, fast staccato notes. Acid: resonant 303-style bassline, sawtooth through a resonant low-pass sweep with accent. · Pitch mapping: the knob's sweep step (0 .. 47) maps exponentially to 110 Hz x 2^(3s/47), i.e. 110 Hz at the bottom to 880 Hz at the top — three octaves, evenly spaced in musical pitch at 0.75 semitones per step, deliberately not snapped to semitones so the knob reads as a continuous sweep. · Tempo: a single fixed 120 BPM clock steps all four mood patterns; the 16th-note arpeggio step is therefore 125 ms. Kept provisional — the user approved 120 BPM but did not ask for it to be adjustable. · Transport is strictly local: the device never starts making sound on its own after a reset, which makes it safe to leave powered in a room.