Page cover

SoundDrop

A revolutionary full-stack web platform that revamps music promotion. SoundDrop converts album art into interactive, audio-reactive 3D visuals, offering artists a unique way to showcase their work.

1. Introduction

Project Overview

SoundDrop is an innovative web application designed to revolutionize the way musicians and artists promote their music online. It combines cutting-edge web technologies to create interactive, audio-reactive visual experiences that serve as unique landing pages for music promotion. SoundDrop allows artists to upload their tracks and cover art, which are then transformed into dynamic 3D visualizations that react in real-time to the music - complete with links to their music on all platforms.

The project aims to provide a platform where artists can create engaging, shareable content that stands out in the crowded digital music landscape. By leveraging the power of modern web browsers, SoundDrop delivers a high-fidelity audio-visual experience without the need for any plugins or additional software.

Visit site ↗

View example demo 1 ↗

View example demo 2 ↗

Implementation

As the sole creator and developer of SoundDrop, my role encompassed the entire development lifecycle, from conceptualization to implementation. This included:

  1. Architecting the overall structure of the application

  2. Developing the front-end user interface using React

  3. Implementing 3D graphics and novel audio-reactive visualizations using React Three Fiber/Three.js and custom WebGL shaders

  4. Setting up the backend infrastructure using Firebase for authentication and data storage

  5. Integrating Stripe's API for payment processing and subscription tracking

  6. Optimizing performance for smooth rendering across various devices

  7. Implementing responsive design for both desktop and mobile experiences

This project showcases my ability to work with complex, interdisciplinary technologies and create a cohesive, user-friendly application that pushes the boundaries of what's possible in web development.

Technologies Used

SoundDrop leverages a modern tech stack to deliver its unique features:

  • React: The core framework used for building the user interface and managing component state.

  • Three.js and React Three Fiber: Used for creating and rendering 3D graphics in the browser.

  • Web Audio API: For processing and analyzing audio in real-time.

  • Firebase: Provides backend services including authentication, database, and hosting.

  • GLSL: The OpenGL Shading Language used for writing custom shaders for advanced visual effects.

This combination of technologies allows SoundDrop to deliver a seamless, interactive experience that bridges the gap between audio and visual art forms. The project demonstrates the potential of web technologies to create rich, immersive applications that were once only possible with native software.


2. Key Features

Interactive 3D Graphics

SoundDrop transforms album cover art into an interactive 3D point cloud using React Three Fiber. This creates a visually stunning representation of the artist's work.

I created a method to process the image data in order to assign a vertices array for the point cloud:

This code iterates through each pixel of the image, creates a color from the RGB values, calculates a weight based on the color, and then creates a vertex with x and y coordinates based on the pixel position, and a z coordinate based on the calculated weight. This effectively turns the 2D image into a 3D point cloud where the "depth" of each point is determined by the brightness of the corresponding pixel

I then convert the array of vertices into a Float32Array for use with Three.js:

Live demo showcasing interactivity and audio-reactivity

Audio Reactivity

One of the most impressive features is the real-time audio reactivity. The point cloud dynamically responds to the music, creating a unique visual experience for each track.

The FFT data is what drives the movement of the point cloud, scaling the Z-axis of pixels based on audio frequency

User Authentication

Firebase is used for secure user authentication, allowing artists to create and manage their accounts.

Customizable Pages

Artists can create personalized pages with custom URLs, adding links to their various streaming platforms.

Responsive Design

The application is fully responsive, ensuring a great user experience on both desktop and mobile devices.

Subscription Model

Stripe's API is integrated for handling payments and managing subscriptions, allowing for premium features and monetization.

Hooking into Stripe's API to handle subscriptions

3. Architecture Overview

In developing SoundDrop, I carefully designed an architecture that could handle complex 3D visualizations, real-time audio processing, and user data management while maintaining a smooth user experience. Here's an overview of the key components:

Frontend Framework

I chose React as the primary frontend framework for its component-based architecture and efficient UI updates. This allowed me to create a modular and maintainable codebase.

Users can edit the visuals with filters and backgrounds, all custom GLSL shaders

3D Graphics Engine

For the core of SoundDrop's visual experience, I integrated Three.js with React Three Fiber. This combination allowed me to create complex 3D scenes declaratively within my React components. I also implemented custom shaders for advanced visual effects:

State Management

While React's built-in hooks are great for local state, I needed a more robust solution for global state management. I implemented Zustand, a lightweight yet powerful state management library:

This allowed me to efficiently manage global state across the application, from audio settings to user preferences.

Backend and Database

For the backend, I chose Firebase for its real-time capabilities and ease of integration. I implemented authentication, data storage, and real-time updates:

Audio Processing

One of the most challenging and rewarding aspects was implementing real-time audio processing. I used the Web Audio API to analyze audio and drive the visualizations:

This architecture allowed me to create a seamless integration between the audio input and the visual output, resulting in the dynamic, music-responsive visualizations that are at the heart of SoundDrop.


4. Code Deep Dives

3D Rendering with React Three Fiber

In SoundDrop, I leveraged React Three Fiber to create dynamic 3D visualizations. One of the key components I developed was the PointCloud, which transforms the album cover into an interactive 3D object:

This component creates a point for each pixel in the album cover, positioning it in 3D space. The z-axis of each point is then modulated based on the audio data, creating the dynamic, music-reactive effect.

Custom Shaders

To achieve more complex visual effects, I implemented custom shaders. Here's a complete example of a custom shader I used in SoundDrop to create a dynamic, audio-reactive background:

This shader creates a dynamic, flowing background that responds to the bass and treble intensities of the audio. The noise function creates organic patterns, while the audio input influences the color mixing and adds a sparkle effect.

To use this shader in my React Three Fiber setup, I created a custom material:

Another live demo with custom kaleidoscope GLSL shader

Audio Processing

Audio processing was crucial for creating the reactive visualizations. I used the Web Audio API to analyze the audio in real-time:

I then used this analyzer in my React components to drive the visualizations:

Firebase Integration

For user authentication and data storage, I integrated Firebase. Here's how I set up the Firebase configuration and implemented user registration:

Image Storage and Retrieval

To handle image uploads and retrievals efficiently, I implemented a system using Firebase Storage. Here's how I managed this process:

When a user uploads a cover art image, I first resize it client-side to ensure consistent performance, then upload it to Firebase Storage:

To display the images, I fetch the download URL and use it in my components:

Subscription Tracking in Firebase

To manage user subscriptions, I utilized Firebase's Firestore database along with Cloud Functions to handle Stripe webhook events. Here's an overview of the process:

When a user subscribes, I store their subscription information in Firestore:

I use a Cloud Function to listen for Stripe webhook events and update the user's subscription status accordingly:

In my application, I check the user's subscription status to determine feature access:

To allow users to add and manage their music platform links, I created a LinkManager component:

SoundDrop was my first and most robust build of a full stack web app. I built it over the course of a few months, pushing myself to come up with novel ways to display the cover arts and create custom filters. It taught me valuable lessons about organizing databases in structured ways and interfacing with payment API's. Check it out for yourself ↓

Visit site ↗

Last updated