iTranslated by AI
Technical Architecture of My Personal App "My Pet Melody"
I have released the following mobile app, so I'm summarizing the technical details here.
I have summarized non-technical aspects in the following article, so please take a look if you're interested.
What kind of app is it?
It is an app that allows you to create original music like the following from cat videos.
Technically, it includes the following mechanisms:
- Analyzes the selected cat video and automatically detects meowing parts.
- Synthesizes the meowing audio with background music (BGM) to generate an audio file.
- Generates a video with a still image from the audio file and a single image.
Selection Policy
I needed to detect cat meows from audio and automatically generate videos, but I had almost no prior knowledge in those areas. Therefore, for everything else, I tried to use technologies I was personally familiar with as much as possible.
Architecture
Mobile App
I implemented both iOS and Android versions using Flutter.
API
The API called from the app uses Cloud Functions.
The reason I am using Cloud Functions directly rather than Cloud Functions for Firebase is that it was easier to call from Cloud Tasks.
I used Flask as the API framework.
Asynchronous Processing
The process for generating videos is executed asynchronously using Cloud Tasks' task queues.
Storage
I am using Cloud Storage for Firebase.
When uploading or downloading videos from the mobile app, I access it directly without going through the API. Therefore, I have set up Storage security rules like the following to restrict each user to reading and writing only within their own dedicated directory hierarchy.
rules_version = '2';
service firebase.storage {
match /b/{bucket}/o {
match /userMedia/{userId}/pieces/{allPaths=**} {
allow read, write: if request.auth != null && request.auth.uid == userId;
}
}
}
Cloud Functions also access it to refer to the videos uploaded by users during the video generation process.
Database
I am using Firebase's Cloud Firestore.
I access it directly without going through the API when referencing data for UI display in the mobile app or when registering tokens for push notifications. Therefore, I have set up Firestore security rules like the following to restrict each user to reading and writing only their own dedicated documents.
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /users/{userId} {
allow read, write: if request.auth != null && request.auth.uid == userId;
}
}
}
When accessing from the mobile app, I establish a real-time update connection so that any data updates in Firestore are immediately reflected in the mobile app's UI.
Cloud Functions also access it to write metadata for generated videos and to retrieve target tokens for sending push notifications.
Meow Detection
I am using the Pydub library.
I tried several libraries using machine learning, but a simple algorithm that detects based on volume level thresholds, as mentioned above, demonstrated the highest accuracy.
Subscription Billing
I am using RevenueCat to implement the billing process on the mobile app side and subscription status checks on the Cloud Functions side.
Authentication
I am using Firebase Authentication.
I use anonymous authentication so that users can use the app's features without creating an account. Additionally, I have enabled Sign-in with Google, X (Twitter), and Apple.
Push Notifications
I am using Firebase Cloud Messaging.
Once a token for sending push notifications is issued in the app, it is registered in Firestore. Also, Cloud Functions retrieves the token from Firestore to send push notifications.
Analytics
I am using Firebase Analytics.
In addition to the default settings that collect data automatically, I also collect screen transition events.
Furthermore, I collect some custom events to analyze at which stage users drop off during the most important sequence of the user experience flow in the app.
Monitoring
I am using Firebase Crashlytics.
In addition to the default settings that collect data automatically, I also collect exception occurrences in Flutter.
App Distribution for User Testing
Firebase App Distribution is used.
Since only a few devices need to have the app installed, for iOS, I distribute it using an Ad Hoc build based on the premise of collecting the UDIDs of the devices to be installed in advance.
Unit Testing
I write unit tests for the API to verify that the recognition score for meow detection has not decreased.
I prepared about six sample videos containing cat meows and quantified whether those meows could be detected without excess or deficiency to create a recognition score.
This value is tested to see if it has changed from the value at the time of initial implementation.
If this value changes due to API modifications or library updates, it may affect the usability of the meow detection feature within the app.
This is a mechanism for early detection of such impacts.
Additionally, I write unit tests only for the parts of the mobile app where complex logic occurs.
Unit tests for the mobile app logic use Flutter's standard testing framework.
E2E Testing
E2E tests are written using Maestro.
I have written only one test case to check if the most important user experience in the app has regressed.
Maestro is very handy and easy to use because, for simple test cases, you only need to list the operations in a YAML file.
Local Development
I use Firebase Local Emulator Suite so that daily development can be completed entirely locally.
The purpose is to reduce the frequency of Google Cloud and Firebase usage during development and cut costs.
When pointing a mobile app to the local emulator, it's necessary to pass the private IP address of the PC on the LAN to the app.
However, since the private IP address changes depending on the PC environment, it's better to avoid writing it in files managed by Git and committing them.
Therefore, I use a mechanism in Flutter that allows passing a file containing variables during build time, and I keep the file containing those variables out of Git's version control.
{
"SERVER_HOST": "192.168.10.13"
}
flutter run --dart-define-from-file "dart-defines.json"
IaC
I use Terraform to automate the construction of the following, including the creation of Firebase projects.
- Creation of Google Cloud projects and associated Firebase projects
- Enabling Firebase Authentication and anonymous authentication
- Enabling Firebase Firestore and setting security rules
- Enabling Firebase Storage and setting security rules
- Creation of task queues in Cloud Tasks
Since the Firebase provider for Terraform is still in beta, there are some parts that are out of reach. Those parts are constructed through manual command execution or operations on the web dashboard.
CI/CD
I am using GitHub Actions to set up the following CI/CD pipeline:
- Static analysis for PRs
- Unit tests for PRs
- Check if builds can be completed for PRs
- Deployment of the user testing app when a tag is pushed
- Automatic deployment of Cloud Functions when a tag is pushed
- Deployment, submission for review, and release of the public app when a tag is pushed
Additionally, I use Fastlane to allow these tasks to be performed with a single command, so I only need to call that command within GitHub Actions. This eliminates the need to write duplicate scripts for local execution and CI/CD.
Discussion