AI as Cross-Culture Music Translator

South Dakota Symphony Orchestra: Artificial Intelligence as a Cultural Translator for Musicians and Audiences: as part of the Lakota Music Project (a partnership with Lakota and SDSO musicians), discover if an AI model can serve as a cultural translator of music thereby influencing how composers create, musicians perform, and audiences engage. 

The AI Development work was provided by Dakota State University’s Beacom College of Computer and Cyber Sciences.

We endeavored to discover if an AI model can serve as a cultural translator between Lakota and western classical music traditions. If so, could the information obtained through the AI model influence how a composer creates, musicians perform, or how audiences engage with this music when they hear it?

The questions we wanted to answer were:

• Can an AI model foster understanding about the similarities and differences between
  western classical and Lakota music.
• Who are the ideal users for the AI model?
• Could the model be used by our concert attendees?

The South Dakota Symphony Orchestra (SDSO) engaged Dakota State University’s (DSU) Compass Program to develop a proprietary AI multi-modal model to give insights and comparison on cultural similarities and differences between Lakota and western classical music. The system would be a LLM (large language model) designed to allow a user to enter a prompt relating to either Lakota, Western classical music or both, and draw comparisons.

The process

Step 1 – Data Gathering and Processing

• The first step was to assemble data to be used and how it would fit into the model to be trained. Ultimately, the model was trained with about 50 Lakota songs, Lakota Music Project compositions, and a limited amount of western classical music.
• Preprocess the data into a format for either non-parametric or parametric model learning.
• Identify the model to be used for the project (the project data is multi-modal)

Step 2 – Building/Testing the model

• The AI model was populated with the data
• Unit testing in collaboration with SDSO stakeholders to validate model learning against ground truths.

Step 3 – Reporting/User front end

• The DSU team builds a user interface (web-based) for model interaction
• DSU and SDSO held regular meetings during the early phase of the project
• One early review of the model during the building/testing phase
• Final unveiling of the model

Artist Reflection

Cultural Infrastructure and Ecology

Our goal was to discover if an AI model could illuminate cultural similarities and differences through music; specifically Lakota songs and western classical music. Our interest in developing an AI model to analyze these two different cultural heritages stems from the SDSO’s evolution in developing the Lakota Music Project. The South Dakota Symphony Orchestra began the Lakota Music Project 20 years ago, with the intention of addressing racial tension through making music together. The Lakota Music Project is a collaboration between Lakota and Dakota musicians (from the Oglala Lakota Sioux and Sisseton Wahpeton Oyate tribes) and SDSO classical musicians. The SDSO has commissioned five works for the Lakota Music Project that combine Lakota singing, Dakota cedar flute playing and classical instruments.

This AI model does not create or produce music. Rather, it analyzes music composed by and performed by artists. The model is only accessible by users with pre-approved email addresses using an authentication password. The model is hosted on a DSU server and not available to the public. It uses a website address for the AI interface.

The user enters a question or prompt, similar to using ChatGPT, Gemini, CoPilot, or Claude. The data sources for this model are performances of Lakota songs, Lakota Music Project compositions, and a limited amount of Western classical music.

What we learned:

1) Our project revealed the sensitivity with which machine learning can determine outcomes from audio recordings. For example, the model explains relative comparisons on percussive sounds, hertz (energy produced), tempo, etc.
We learned that the model responds most effectively when prompts are closely aligned with the audio-based nature of its training data. Questions that focus on sound—such as rhythm, texture, energy, pitch, instrumentation, or acoustic similarity—produce clearer and more useful results than broad prompts about history, language, or cultural context. For example, the model is not adept at translating written or spoken language or providing historical/contextual information.

Effective prompts tended to:

• Ask the model to compare specific sonic qualities between Lakota and Western classical examples.
• Recognize the model’s limits as an audio-analysis tool rather than a source for translation, historical explanation, or cultural interpretation.

2) We also learned the importance of verifying the data used to train the model. Although a substantial body of western classical music had been identified for inclusion, not all of those recordings were incorporated into the final dataset. As a result, the model was trained on a much larger collection of Lakota music than Western classical music.

This imbalance did not prevent the model from functioning, but it likely reduced the richness and robustness of the comparisons that could be made between the two traditions. This underscores the importance of dataset management as a core component of AI development.

The project would have benefited from a more formal process for auditing the training dataset and confirming that all intended recordings were included prior to model training.

The development team has expressed a willingness to incorporate missing western classical recordings as resources permit, pending intellectual property rights and permissions, which emerged as an important area of learning throughout the project.

In retrospect, one of the project’s most valuable lessons was that data preparation, documentation, verification, and rights management are as important as the model architecture itself. Future work would incorporate a structured dataset audit at key project milestones to ensure completeness, improve reproducibility, and strengthen confidence in the resulting analysis.

As a research and exploratory effort, the project provided valuable insight not only into the potential of AI as a tool for cultural analysis, but also into the practical workflows required to develop such systems responsibly and rigorously.

3) The user query is run through a MERT model to analyze the music (Music undERstanding model with large-scale self-supervised Training). The MERT model runs a comparison between Lakota and classical music.

Each query is analyzed using the following three factors:

1. Retrieving 10 most relevant pieces
2. Extracting acoustic features and MERT embeddings
3. Generating the cross-cultural analysis. User testing revealed a tradeoff between analytical depth and responsiveness. To generate a response, the model evaluates each prompt against three separate factors, resulting in multiple layers of analysis for every query. This process can require several minutes to complete, making the system noticeably slower than users may expect from other AI tools. Although the slower performance reflects the complexity of the model’s analytical approach, improving speed and usability would be a priority in future iterations of the project.
4. Our plan is for the composer, Derek Bermel, and Lakota Singer, Emmanuel Black Bear, to use the model in preparation for a premiere of a newly orchestrated version of their collaborative work, “Lakota Refrains”, on November 20, 2026. The original project plan envisioned that students attending the concert would use the model and share their experiences with us. As testing progressed, however, it became clear that the current user experience may not be well suited to that purpose. Each query is evaluated across multiple analytical factors, and users often wait several minutes for a response. Unless we can improve the functionality of the model for wider use, we will narrow the users to select musicians and composers. We are therefore reevaluating whether student testing in its current form would be productive or engaging. Although the model is capable of generating meaningful insights, the pace of interaction risks making the experience feel laborious rather than exploratory. Improving system performance and usability would be an important next step before broader student
   engagement.

AI Transparency

An unexpected finding of the project was that legal and cultural questions surrounding training data were among the most difficult aspects of the work. While Emmanuel Black Bear gave us permission to use his recordings, much of the western classical repertoire was drawn from public-domain musical works, it was not practical, and in some cases not possible, to identify and obtain authorization from every potential rights holder represented in the dataset. The project also included. Certain Lakota ceremonial songs that were publicly accessible through online sources. These recordings were available for public listening and download, but their availability did not necessarily clarify questions of copyright ownership, cultural authority, or appropriate use in AI research. The team therefore had to navigate a landscape in which the relevant law remains unsettled and best practices are still emerging.

We view this not as a shortcoming of the project but as an important result. Any future effort to build culturally informed AI systems using music as the dataset will need to address questions of rights, consent, provenance, and cultural stewardship alongside the technical challenges of model development.

The project’s purpose was not to generate, reproduce, or imitate music. Rather, the model was developed as a research tool to explore whether machine learning could identify and differentiate musical characteristics associated with western classical and Lakota musical traditions, and whether such distinctions might contribute to future forms of cultural understanding or translation through music. In this sense, the process of assembling an appropriate and ethically responsible training corpus became one of the project’s most significant lessons.

It is also important to note that the resulting model is not publicly available. The model resides on a proprietary server and can be accessed only by authorized users through password-protected accounts. While we recognize that restricted access does not resolve questions of copyright or cultural stewardship, it provided some assurance that the source materials would not be redistributed through the project, and that use of the model would remain limited to the specific research and educational purposes for which it was developed.

Helpful AI copyright resource: https://www.humanartistrycampaign.com/

Transparency

Models / Tools Used

yes

Dataset Sources

yes

Consent / Stewardship Plan

yes