Knowledge Base: Music Data Exchange Format (MDEF)
The Music Data Exchange Format (MDEF)
Specification Version 0.3 (August 2009):
Abstract:
The Music Data Exchange Format (MDEF) is a free, open and portable XML-based format created by Band Metrics (an Indie Music, Inc. company) for the purpose of sharing social data relative to musicians and bands, because presently, there does not exist such a format. Music data that can be exchanged using this format can range from the number of song plays a band has on MySpace, to the number of fans they have on Imeem, or the number of comments they have on PureVolume, or the number of subscribers they have on their YouTube Channel, etc.
This document outlines MDEF, but because the format is in its infancy, this document will continue to grow and be refined.
Table of Contents:
1.0 Introduction
1.1 History
1.2 Why
1.3 Name
1.4 Pronunciation
1.5 Logo
1.6 Status
2.0 Use
2.1 Getting data from Band Metrics
2.1.1 Example of XML from an artist page
2.1.2 XML Elements
2.1.2.1 Band id
2.1.2.2 lastUpdated
2.1.2.3 Profile
2.1.2.4 Scores
2.1.2.5 Stats
2.1.3 Data Restrictions
2.2 Getting Data To Band Metrics
2.2.1 Identity Verification
2.2.2 Query Verification
2.2.3 Query Specification
2.2.4 Query Example
2.2.5 Exposing XML
2.2.6 Data Restrictions
3.0 Questions, Comments and Suggestions
4.0 Brief FAQ
5.0 License and Attribution
6.0 Cancellation
7.0 Abuse
8.0 Notices
9.0 Revisions
10.0 Contact
11.0 Creative Commons License
12.0 Copyright Notice
12.1 Copyright Statement
1.0 Introduction:
With the enormous amounts of data being created by artists of all sizes each day (e.g. plays, downloads, likes, faves, sharing, recommending, etc.), it's readily apparent that there needs to be a way to organize and share data relative to musicians and bands effectively and efficiently. With that said, there is presently no free, open and portable standard format for exchanging artist data, and while Band Metrics will soon offer RSS data updates, RSS is not sufficient or optimal. As such, Band Metrics decided to create The Music Data Exchange Format (MDEF) since measuring artist data is what we do, and we wanted to help other companies and individuals collaborate on this effort via our soon-to-be-released API.
1.1 History:
Given the challenges of collecting online data relative to bands and musicians, Band Metrics began discussing the idea of creating an open API architecture in December of 2008, and began drafting documentation in January of 2009. Development on the Band Metrics API framework began in March of 2009. The Music Data Exchange Format (MDEF) developed out of this process so there could be a standard way to send/receive data to/from Band Metrics, as we encountered too many variations in the way data is being expressed online.
1.2 Why:
By creating a standard way to share artist data, The Music Data Exchange Format (MDEF) can enable any music tech company to work closely with another one for the benefit of the entire music community.
1.3 Name:
The Music Data Exchange Format (MDEF) name was coined by a former Band Metrics contractor, Bryan Johnson.
1.4 Pronunciation (mDef):
Ian Rogers of Topspin Media came up with a great way to pronounce the format when he referred to it as "mDef" during the SanFran MusicTech Summit on May 18, 2009.
1.5 Logo:
The Music Data Exchange Format (MDEF) logo was developed by Band Metrics' designer, Jared England.
1.6 Status:
The initial schema for the Music Data Exchange Format (MDEF), version 0.1, was released on May 18, 2009 at the SanFran MusicTech Summit. While the format is presently operational, it's in the early stages of development.
2.0 Use:
Because this format is in its infancy, there are only two limited ways to use it at present: 1) retrieving data from Band Metrics and 2) making music data available to Band Metrics so our system can collect your artist related data and process it in our analytical engine. Below, we have outlined the basics for both.
2.1 Getting data from Band Metrics:
Initially, individuals and companies can pull artist data with an HTTP request and an XML response. The request is sent in the format of: http://www.bandmetrics.com/artist/Artist_Name/xml. The XML data can also be readily obtained from any artist using Band Metrics by clicking on the XML button on the band's public page.
2.1.1 Example of XML from an artist page (The Dave Matthews Band):
<profile>
<name>Dave Matthews Band</name>
<description>a band description goes here</description>
<links>
<link rel="website">http://davematthewsband.com</link>
<link rel="blog">http://www.davematthewsband.com/#/news</link>
<link rel="facebook">http://www.facebook.com/davematthewsband</link>
</links>
<location>
<city>Charlottesville</city>
<state>Virginia</state>
<country>United States</country>
</location>
</profile>
<scores rank="platinum">
<score type="pageViews">116.7</score>
<score type="followers">134.5</score>
<score type="socialplays">112.1</score>
<score type="comments">132.3</score>
<score type="radioplays">102.3</score>
<score type="hotness">120.8</score>
<score type="fandom">94</score>
<score type="brand">120.8</score>
</scores>
<stats>
<feed id="myspace">
<plays>0</plays>
<fans>343805</fans>
<conversations>88380</conversations>
<viewers>10354658</viewers>
</feed>
<feed id="imeem">
<plays>212323</plays>
<fans>42140</fans>
<conversations>1423</conversations>
<viewers>75321</viewers>
</feed>
<feed id="ilike">
<plays>2517</plays>
<fans>1385173</fans>
<conversations>1610</conversations>
<viewers>nodata</viewers>
</feed>
<feed id="soundcloud">
<plays>nodata</plays>
<fans>nodata</fans>
<conversations>nodata</conversations>
<viewers>nodata</viewers>
</feed>
<feed id="lastfm">
<plays>20924424</plays>
<fans>635515</fans>
<conversations>0</conversations>
<viewers>0</viewers>
</feed>
<feed id="purevolume">
<plays>0</plays>
<fans>0</fans>
<conversations>0</conversations>
<viewers>0</viewers>
</feed>
<feed id="youtube">
<plays>97834</plays>
<fans>24501</fans>
<conversations>857</conversations>
<viewers>39000</viewers>
</feed>
</stats>
2.1.2 XML Elements:
The elements below comprise the initial format for sending and
receiving data.
2.1.2.1 Band id:
The id is the Artist's unique name (set by the artist at signup)
which we use to create both the public band page and the link to
the XML for the API. For example:
http://www.bandmetrics.com/artist/davematthewsband
http://www.bandmetrics.com/artist/davematthewsband/xml
2.1.2.2
lastUpdated:
The lastUpdated variable is a unix timestamp for the last time the
information for the band was changed.
2.1.2.3 Profile:
The profile section contains the broader non-statistical and
non-event based data for the band:
<profile>
<name>Dave Matthews Band</name>
<description>a band description goes here</description>
<links>
<link rel="website">http://davematthewsband.com</link>
<link rel="blog">http://www.davematthewsband.com/#/news</link>
<link rel="facebook">http://www.facebook.com/davematthewsband</link>
<links>
<location>
<city>Charlottesville</city>
<state>Virginia</state>
<country>United States</country>
<location>
</profile>
Please note, while most of this is pretty straight forward,
there are a few things that should be noted:
The description field is only intended as a brief description
of the band and is limited to 140 characters. The links
block can contain 0 to n links and each link must have a rel value.
The rel value is a one word description of what is being linked to.
Additionally, the URL will be a full URL, not relative.
The location block contains city, state, and country.
Geo-location data will be added later.
2.1.2.4 Scores:
The scores block contains the calculated scores for the Band
Metrics Ranking system. The first line is: <scores
rank="platinum">. This tells you the band's overall score
level.
Each of the subsequent score lines consists of a score type (page,
followers, pageviews, comments and brand) and a numerical score
value: <score type="plays">112.1</score>
2.1.2.5 Stats:
The stats block contains the raw data from each of the feeds we
track for the band. There can be 0 to n feed blocks in the stats
block. Additionally, if we're not collecting any data from a
particular feed, for example, the artist does not have a PureVolume
account, than the stats block will be "no data." This is an
example.
<stats>
<feed id="myspace">
<plays>0</plays>
<fans>343805</fans>
<conversations>88380</conversations>
<viewers>0</viewers>
</feed>
<feed id="purevolume">
<plays>no data</plays>
<fans>no data</fans>
<conversations>no data</conversations>
<viewers>no data</viewers>
</feed>
<feed id="lastfm">
<plays>20924424</plays>
<fans>635515</fans>
<conversations>0</conversations>
<viewers>0</viewers>
</feed>
<stats>
So in the example above, the band currently has feeds for
myspace and lastfm. Each feed block will contain plays, fans,
conversations and viewers (even when the actual value may be 0). In
the case of PureVolume, the artist does not have an account, which
produces a message on an artist's profile of "no data." if you
believe this is an error, please contact support via email at
help@bandmetrics.com.
At the end of the stats block, we close out the band data with
</band> which finishes out the XML. Additionally, and as you
are aware, the potential methods for retrieving and parsing the XML
are dependent on your programming language of choice, and beyond
the scope of this document. However we will post relevant examples
as they become available.
2.1.3 Data
Restrictions:
Time based data will be delivered as a UNIX
timestamp and all numerical values will be delivered without
formatting (no commas). The band description will be no more than
140 characters.
2.2 Making data
available to Band Metrics:
Making music data available to Band Metrics so our system can
collect your artist related data and process it in our analytical
engine is a bit more difficult, as we have to ensure the security
of our system, the quality of data, resource use on our system and
the applicability of the data, as well as other aspects. However,
we will endeavor to work with any data provider as we would like to
accommodate all requests.
2.2.1 Identity
Verification:
With that said, our initial specification will require that we
verify your identity and data source. To initiate the process,
please send an email to developers@bandmetrics.com
with the following information:
Your Name
Company/Organization Name
Website
Your Email Address
Your Telephone Number
Your Mobile Number
Your Business Address
Your Mailing Address (if different)
A brief statement proposing why you would like to send data to Band
Metrics
2.2.2 Query
Verification:
Once your identity and applicability has been confirmed and
authorized, verified data providers must generate a URL that
conforms to the Elements listed in section 2.1.2. That is, you will
need to provide Band Metrics a URL where we can post a value and
receive an XML reply in the same format as our own XML API reply
detailed in the Getting Data From Band Metrics section (see 2.2).
This will ensure interoperability.
2.2.3 Query
Specification:
The query we send will be in the form:
http://The_URL_You_Provide/?query=artist_name&start=#&limit=#
The artist_name will be the unique identifier for the artist in
your system. The start and limit allow you to set appropriate
limits for your system and makes sure that we grab the correct data
set we need.
The data you return will match our previously described data
output, with one exception; Given that a query can easily return
more than one result, the returning XML will need to be wrapped in
an overall bands block, something like so:
<bands error=string totalCount=int numResults=int
start=int>
<band>blah blah blah</band>
<band>blah blah blah</band>
<band>blah blah blah</band>
</bands>
The error is optional and should be used when a query returns no
results. The string should provide information that will help us
determine why the query failed and some idea of how to correct
it.
The totalCount should provide us with the total number of results
the query returned.
The numResults should provide us with the total number of results
that you are sending back to us. This is useful for when your
system only returns a specified number of results per query.
The start tells us what record we are starting with. This allows us
to make multiple queries if your system requires this.
2.2.4 Query Example:
If we make a query that returns 500 results from your system and
you have a limit of 100 results maximum per query, your bands block
will look like this:
<bands totalCount=500 numResults=100 start=1>
This lets our system know that the next request we make should be
for:
http://The_URL_You_Provide/?query=artistname&start=101&limi...
and your next reply would start with:
<bands totalCount=500 numResults=100 start=101>
If an error where to occur, the band block would look something
like this:
<bands error="The query has timed out. The database server may
be busy">
</bands>
Once your organization has been verified via the above process and
cleared to send Band Metrics data, we will then begin processing
your data at the earliest possible convenience.
2.2.5 Exposing XML:
Alternatively, it's possible to exchange data if you expose artist
data via a unique url and conform to the MDEF format. For example,
exposing concert/show data for artists via individual artist pages
using an artist's myspace id as a way to verify the correct
assocaition, like the following:
http://www.companyname.com/artist/artist_name/myspaceid/show_announ...
<artist_myspaceid>61245820</artist_myspaceid>
<show_date>January 15, 2010</show_date>
<venue_name>Smith's Olde Bar</venue_name>
<venue_address>1578 Piedmont Ave</venue_address>
<venue_city>Atlanta</venue_city>
<venue_state_region>Georgia</venue_state_region>
<venue_postal_code>30324</venue_postal_code>
<venue_country>United States</venue_country>
<venue_telephone>404-875-1522</venue_telephone>
2.2.6 Data
Restrictions:
Time based data will be delivered as a UNIX timestamp and all
numerical
values will be delivered without formatting (no commas). The
returning XML should conform to current standards as defined by the
W3C: http://www.w3.org/XML/
3.0
Questions, Comments and Suggestions:
We're always looking for
ways to improve, so if at anytime you have questions, comments
and/or suggestions about this format, please post them here:
http://help.bandmetrics.com/discussions/developers
This will allow for a collaborative effort in creating the most
useful and efficient format
4.0 Brief FAQ:
4.1 Does this Music Data Exchange Format (MDEF) provide access to
all the data Band Metrics collects?
No, but we plan to expand access frequently as our resources
allows, so please keep in touch
4.2 Does the MDEF provide access to artist charts and graphs within
Band Metrics?
Not at this time, but we're considering as part of a future
offering
4.3 How often is the data relative to the artists within Band
Metrics updated?
Once per hour, but as we enhace our system we would like to get to
around every 30 minutes or better
4.4 What data sources do you plan to add to Band Metrics analytical
engine?
iLike, Facebook, Twitter, Reverb Nation, Ustream, Muesbin, Hypem,
as well as many others are on the to do list for the near
future
4.5 What can I do with the data collected by Band Metrics and made
freely available to the public?
Anything you want - from mashups with companies like flickr and
Google Maps to your own web app.
4.6 If my company produces artist data different than what Band
Metrics is collecting, can we still provide that data to Band
Metrics for using within its analytical engine?
Depends, you would need to speak with us first by sending an email
to developers@bandmetrics.com
4.7 Are there any companies using mDef?
Yes! SoundCloud is and the next version of their API will have
elements of the mDef format in it.
5.0 License and
Attribution:
There are no license requirements to use our Music Data Exchange
Format (MDEF). In fact, we encourage open use, however, we do ask
that you kindly attribute your data collection to the Music Data
Exchange Format (MDEF), when and where applicable (see copyright
information below).
6.0 Cancellation:
At anytime, Band Metrics reserves the right to cancel your ability
to send data to Band Metrics for data analysis.
7.0 Abuse:
If at anytime, Band Metrics determines that you have attempted to
abuse our system and/or users via The Music Data Exchange Format
(MDEF), your ability to send data to Band Metrics will be
revoked.
8.0 Notices:
This format may be updated at anytime by Band Metrics, so please
stay in touch, and we will endeavor to let all of our developers
know when we update our Music Data Exchange Format (MDEF).
9.0 Revisions:
version 0.1: initial release
version 0.2: additional attributes, mDef logo
10.0 Contact:
For additional information, please send us an email to developers@bandmetrics.com.
11.0 Creative
Commons License:
The Music Data Exchange Format (MDEF) is licensed under the
Creative Commons Attribution 3.0 United States License. To view a
copy of this license, visit http://creativecommons.org/licenses/by/3.0/us/
or send a letter to Creative Commons, 171 Second Street, Suite 300,
San Francisco, California, 94105, USA.
12.0 Copyright
Notice:
The Music Data Exchange Format (MDEF) Copyright (C) Indie Music,
Inc. (2009). All Rights Reserved.
12.1 Copyright
Statement:
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise
explain it or assist in its implementation may be
prepared, copied, published and distributed, in whole
or in part, without restriction of any kind, provided
that the above copyright notice be included on all
such copies and derivative works.
Additionally, this document and the information contained herein is
provided on an "AS IS" basis and INDIE MUSIC,
INC. DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO ANY WARRANTY THAT THE
USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR A PARTICULAR PURPOSE.