Music Models

Overview of the capabilities of AIML API audio / music models.

Overview

Our API features the capability to generate audio. With this API, you can create your own music, speech, and any audio experience from your prompt and imagination.

We support multiple music models. You can find the complete list along with API reference links at the end of the page.

Quick Code Example

Here is an example of generation an audio file based on a sample and a prompt using the music model minimax-music from MiniMax.

Full example explanation

As an example, we will generate a song using the popular minimax-music model from the Chinese company MiniMax. As you can verify in its API Reference, this model accepts an audio sample as input—extracting information about its vocals and instruments for use in the generation process—along with a text prompt where we can provide lyrics for our song.

We used a publicly available sample from royalty-free sample database and generated some lyrics in Chat GPT:

Side by side, through thick and thin, &#xNAN;With a laugh, we always win. &#xNAN;Storms may come, but we stay true, &#xNAN;Friends forever—me and you!

To turn this into a model-friendly prompt (as a single string), we added hash symbols and line breaks.

'''\ ##Side by side, through thick and thin, \n\nWith a laugh, we always win. \n\n Storms may come, but we stay true, \n\nFriends forever—me and you!##\ '''

A notable feature of minimax-music model is that sample uploading/voice analysis + music generation and retrieving the final audio file from the server are done through separate API calls. (AIML API tokens are only consumed during the first step—i.e., the actual music generation.)

You can insert the contents of each of the two code blocks into a separate Python file in your preferred development environment (or, for example, place each part in a separate cell in Jupyter Notebook). Replace <YOUR_API_KEY> in both fragments with the AIML API Key obtained from your account.

Next, run the first code block. If everything is set up correctly, you will see the following line in the program output (the specific numbers, of course, will vary):

Generation: {'id': '906aec79-b0af-40c4-adae-15e6c4410e29:minimax-music', 'status': 'queued'}

This indicates that the file upload and our generation has been queued on the server (which took 4.5 seconds in our case).

Now, copy this id value (without quotation marks) and insert it into the second code block, replacing <GENERATION_ID>. Now, we can execute the second code block to get our song from the server.

Processing the request on the server may take some time (usually less than a minute). If the requested file is not yet ready, the output will display the corresponding status. Try waiting a bit and rerun the second code block. (If you're comfortable with coding, you can modify the script to perform this request inside a loop.)

In our case, after three reruns of the second code block (waiting a total of about 20 seconds), we saw the following output:

Generation: {'id': '906aec79-b0af-40c4-adae-15e6c4410e29:minimax-music', 'status': 'completed', 'audio_file': {'url': 'https://cdn.aimlapi.com/squirrel/files/koala/Oa2XHFE1hEsUn1qbcAL2s_output.mp3', 'content_type': 'audio/mpeg', 'file_name': 'output.mp3', 'file_size': 1014804}}

As you can see, the 'status' is now 'completed', and further in the output line, we have a URL where the generated audio file can be downloaded.

Listen to the track we generated below the code blocks.

The first code block (sample uploading and music generation):

# 1st code block
import requests


def main():
    url = "https://api.aimlapi.com/v2/generate/audio"
    payload = {
        "model": "minimax-music",
        "reference_audio_url": 'https://tand-dev.github.io/audio-hosting/spinning-head-271171.mp3',
        "prompt": '''
##Side by side, through thick and thin, \n\nWith a laugh, we always win. \n\n Storms may come, but we stay true, \n\nFriends forever—me and you!##
''',   
    }
    # Insert your AIML API Key instead of <YOUR_AIMLAPI_KEY>:
    headers = {"Authorization": "Bearer <YOUR_AIMLAPI_KEY>", "Content-Type": "application/json"}

    response = requests.post(url, json=payload, headers=headers)
    print("Generation:", response.json())


if __name__ == "__main__":
    main()

The second code block (retrieving the generated audio file from the server):

# 2nd code block
import requests


def main():
    url = "https://api.aimlapi.com/v2/generate/audio"
    params = {
        # Insert the id from the output of the 1st code block, instead of <GENERATION_ID>:
        "generation_id": "<GENERATION_ID>",
    }
    # Insert your AIML API Key instead of <YOUR_AIMLAPI_KEY>:
    headers = {"Authorization": "Bearer <YOUR_AIMLAPI_KEY>", "Content-Type": "application/json"}

    response = requests.get(url, params=params, headers=headers)
    print("Generation:", response.json())

if __name__ == "__main__":
    main()