Today I have been playing a lot with a plugin for Jellyfin called AudioMuse AI. It’s a tool that uses the actual audio features of your music library, including things like tempo, tone and timbre to profile your music library. It uses this information to make recommendations about similar tracks, genres and styles. It seems to be a pretty powerful tool and so far I’ve been quite impressed even though I’ve only had time to play with it a little bit. ## Speeding Up The Analysis Step When you first start the system, it runs a sonic analysis over all of your library to capture information about the songs in a database. I noticed that it was taking a very long time to analyze my music library. When I read the logs I saw that it was running TensorFlow models to provide analysis of the music, but was not making use of my GPU. I spent a couple of hours adding GPU support to the docker images so that I could speed up the processing time. I managed to get it working with my GPU in the end and I sent a pull request to the project. The analysis step is still running and it’s taking a while but songs are processed in about 4s instead of 30s. ## Generating Smart Playlists AudioMute AI is still fairly young software. It currently has its own standalone web UI that allows you to generate playlists and send them back to Jellyfin. There are a few different modes that you can generate playlists in. ### Chat Playlists In this mode, you ask the system for a particular style or theme using written English and it uses a language model to convert your question into an SQL query for the database. You can use local models running on your own system rather than having to send your query off to OpenAI or Google. I found that it works quite well with Olama and Mistral Small 3.2. I did try it with some of the smaller llama models but I found that they were less able to generate sensible SQL queries. ### Similar Tracks Another way to generate playlists is to find sonically similar tracks. You start by entering a song you like into the form and the system finds songs with a similar sound. I was quite pleased and impressed by the results of this approach. ### Other Modes You can create a ‘sonic fingerprint’ which takes your listening history and tries to find other tracks in your library that are “similar”. I believe this is based on the centroid vector of all the songs that you’ve listened to before. There’s also a musical “path” feature which allows you to build a playlist that transitions between two songs. I guess you could start with something easy going and end up with something quite intense and fast (or vice versa). I haven’t worked out how to make this work yet, it just gives me errors. ## Jellyfin Integration At the moment integration is minimal. There is a plugin that allows Jellyfin to ping your AudioMuse AI server and refresh the index periodically but currently no direct integration of the playlist generation stuff. I guess it will get better over time. ## Conclusion I’ve had a fun afternoon messing with this stuff and some of the playlists it’s generated are really good. I’m looking forward to experimenting with the “chat” playlists more and seeing how the software develops over time.

Today I have been playing a lot with a plugin for Jellyfin called AudioMuse AI. It’s a tool that uses the actual audio features of your music library, including things like tempo, tone and timbre to profile your music library. It uses this information to make recommendations about similar tracks, genres and styles. It seems to be a pretty powerful tool and so far I’ve been quite impressed even though I’ve only had time to play with it a little bit. ## Speeding Up The Analysis Step When you first start the system, it runs a sonic analysis over all of your library to capture information about the songs in a database. I noticed that it was taking a very long time to analyze my music library. When I read the logs I saw that it was running TensorFlow models to provide analysis of the music, but was not making use of my GPU. I spent a couple of hours adding GPU support to the docker images so that I could speed up the processing time. I managed to get it working with my GPU in the end and I sent a pull request to the project. The analysis step is still running and it’s taking a while but songs are processed in about 4s instead of 30s. ## Generating Smart Playlists AudioMute AI is still fairly young software. It currently has its own standalone web UI that allows you to generate playlists and send them back to Jellyfin. There are a few different modes that you can generate playlists in. ### Chat Playlists In this mode, you ask the system for a particular style or theme using written English and it uses a language model to convert your question into an SQL query for the database. You can use local models running on your own system rather than having to send your query off to OpenAI or Google. I found that it works quite well with Olama and Mistral Small 3.2. I did try it with some of the smaller llama models but I found that they were less able to generate sensible SQL queries. ### Similar Tracks Another way to generate playlists is to find sonically similar tracks. You start by entering a song you like into the form and the system finds songs with a similar sound. I was quite pleased and impressed by the results of this approach. ### Other Modes You can create a ‘sonic fingerprint’ which takes your listening history and tries to find other tracks in your library that are “similar”. I believe this is based on the centroid vector of all the songs that you’ve listened to before. There’s also a musical “path” feature which allows you to build a playlist that transitions between two songs. I guess you could start with something easy going and end up with something quite intense and fast (or vice versa). I haven’t worked out how to make this work yet, it just gives me errors. ## Jellyfin Integration At the moment integration is minimal. There is a plugin that allows Jellyfin to ping your AudioMuse AI server and refresh the index periodically but currently no direct integration of the playlist generation stuff. I guess it will get better over time. ## Conclusion I’ve had a fun afternoon messing with this stuff and some of the playlists it’s generated are really good. I’m looking forward to experimenting with the “chat” playlists more and seeing how the software develops over time.