Maintaining a Community Based Music Website with Open Source Software Open Source Developers Conference 27 th  November 2009
In the beginning… Newcastle has a thriving music scene Members of the community lacked cohesion No support from commercial radio .com excitement in 1999
 
 
Newcastle Music Directory www.newcastlemusic.com Online since 1999 Over 4500 local artists listed Focuses on music from the Hunter Valley and Central Coast Site has over 8000 pages Updated frequently Sections include: News Links Videos Gig Guide
TE Archive Online since mid 2002 Site is located at 15min.org (ie. 15 minutes of fame) Features articles from the TE liftout from May 1998 to March 2006 Over 13,000 pages Over 6,500 images
Geographical Focus Focuses solely on musical artists and websites from Newcastle and the surrounding regions Allows the site to be comprehensive
Geographical Focus
Why “Local” Music? You can talk to the musicians after the gig
Why “Local” Music? You can talk to the musicians after the gig The musicians will play weekly/monthly in your area
Why “Local” Music? You can talk to the musicians after the gig The musicians will play weekly/monthly in your area CDs are cheaper
Why “Local” Music? You can talk to the musicians after the gig The musicians will play weekly/monthly in your area CDs are cheaper Support the economy of your local region
Why “Local” Music? You can talk to the musicians after the gig The musicians will play weekly/monthly in your area CDs are cheaper Support the economy of your local region Local musicians don’t mime
 
Why Do This For Free? From Wednesday’s keynote talk: Volunteers are “not motivated by financial or material gain” Volunteers are “motivated by ideology”
Volunteering Many local musos effectively volunteer their time for their passion Much of the local music industry is run on goodwill
Aims Focus strictly on the region around Newcastle, to the point of actively excluding news from other areas
Aims Provide the Newcastle region with a musical identity of its own that bands and artists are proud to identify with.
Aims Provide the region with a comprehensive list of current gigs and events.
Aims Provide a comprehensive list of links to locally based websites that relate to the music scene.
Aims Provide up-to-date and informative profiles for as many local bands and artists as possible.
Aims Provide maps and information about local pubs, clubs and musical venues.
Aims Provide an archive historical information about the local music scene for long term reference.
Aims Provide a unified voice to highlight the musical talent of the region to people from other areas.
Aims Meet and collaborate with other people interested in the local music scene
Looking Back … Started compiling links and information about the music scene in 1999 Have published over 20,000 pages about the local music industry on the internet Have volunteered over 7000 hours to the local music scene Received an award from in October 2005 for volunteer services to the local music industry Involvement in local music festivals and competitions All while working full time as an Engineer
Volunteers Award in 2005
 
Effects of the Regional Focus Never any argument about what should be included on the site Restricted market – only really of relevance to the people of Newcastle (~1million people) Allows the website to become vwery comprehensive (~80% coverage?)
Types of Information Bands and Artists Radio Larger Bands and Orchestras - Marching Bands, Orchestras, Choirs Services Music Shops
Types of Information Recording Studios Mobile Discos  Music Organisations  Record Labels Production Tuition
Types of Information Media Events  Managers and Agents Sound and Lighting Promotion Photography
Sources of Information Free webhosting services such as Geocities etc. Mp3.com Their own fully hosted domains mp3.com.au www.abc.net.au/newcastle as part of the ABC Newcastle Music Awards PureVolume.com MySpace.com Blogs on services such as blogger.com Press releases
Sources of Information Newspapers flickr.com Triple J Unearthed YouTube.com Booking agents and promoters websites Facebook Google Calendars Twitter ABC Contribute Word of mouth
Link to External Websites Often the artist will maintain a bio/gig guide etc on another website newcastlemusic.com has no login system
Automated Aggregation of Information More recently websites have allowed access to information via: RSS API Copying information from external websites (with the artists permission)
Embedded YouTube Video Bandwidth supplied by YouTube Information retrieved from YouTube via  RSS Playtime Author Upload date About 2000 videos are embedded in the site
RSS Really Simple Syndycation Many websites automatically generate RSS “feeds” when you update them Livejournal MySpace Blogspot Youtube Flickr etc. RSS allows people to  subscribe  to your blog Alerts people that you have updated your website without them visiting the site. Is a form of XML (eXtensible Markup Language)
Artists Blog Entries Automatically brought into the site daily via RSS feeds Subscribed to over 3500 myspace blogs from the Newcastle music industry Provides an excellent source of up to date news about the local music scene
The Gig Guide Published around 1300 local events per month An additional 500 events from outside the region on the artists profiles Significant effort is made to filter out the local gigs
Number of Local Gigs Published
 
The Gig Guide Database Required a complete list of local venues and artists Compilation of such lists took over 5 years Required significant local knowledge Currently contains: 650 venues 4500 artists
Venues from the Newcastle Region Indexed using Google Maps
Venue Locations
Venue Locations
Sources of Gig Information Booking agents Venues Band Homepages Google Calendar Various other sources
Maintaining the Gig Guide Identifying and removing duplicate entries
Maintaining the Gig Guide Identifying and removing duplicate entries Catching misspelled band or venue names
Maintaining the Gig Guide Identifying and removing duplicate entries Catching misspelled band or venue names Deleting cancelled gigs from the database
Maintaining the Gig Guide Identifying and removing duplicate entries Catching misspelled band or venue names Deleting cancelled gigs from the database Ensuring that the correct band and venue are linked to a particular event in the database
Maintaining the Gig Guide Identifying and removing duplicate entries Catching misspelled band or venue names Deleting cancelled gigs from the database Ensuring that the correct band and venue are linked to a particular event in the database Specifying the time of an event if not otherwise stated.
Updating Gig Information Identify source Disable all future gigs from that source For each gig: Re-enable if already found in database Add new event if not found This allows for cancellations/modifications
Maintaining the Gig Guide Database maintenance is not performed on the webserver Uploaded twice a week Much more stable and maintainable as a result
Gig Archive Over 50000 gigs from around the region All gigs link to artist profile and map of venue Archiving started in November 2006
Open Information Local booking agents and promoters are protective about their business practices Which venues they represent How often venues have entertainment Contact details for venues and artists
Open Information Local booking agents and promoters are protective about their business practices Which venues they represent How often venues have entertainment Contact details for venues and artists
Bringing the Community Together
Open Source Software All websites built on open source software Apache MySQL Wordpress Various libraries (Magpie RSS etc etc)
Open Source Software Allowed the management these websites and the data behind them on a shoestring
In for the long haul … Open Source technologies allow flexibility and continuity
The Stickybeak In use on all websites since 2002 Released in 2004 Operates in a similar way to Google Analytics Available from www.stickybeak.com
Newcastle Music Directory Page Requests Per Day
Silverchair Profile Requests
Artists Profile Requests
Section Usage
Sitemap
Web vs Print Media Internet and print media very different types of content Advertising in music print media often considered content on music websites Case Study: Reverb Street Press
Reverb Street Press Content Street Press content was analysed during late 2007 Approximately 10% of the space was devoted to local artists Over 40% of the page space contained advertisements
 
 
Advertising
Conclusion Open source technologies have allowed the Newcastle Music Directory to grow in an adaptive and economical manner over a sustained period of development.
Conclusion The result is a website which provides relevant information about the music scene to the people of the Newcastle region.
http://www.newcastlemusic.com/ twitter.com/newcastlemusic
 
Local Music on the Internet Library of over 13,000 songs Sourced from: JJJ Unearthed Mp3.com.au MySpace ABC 1233 Awards All are indexed by: File type Stream/download Stereo/mono Play Time File Size Sample Rate Bit Rate
Newcastle Survey February 2007 515 Download MySpace 2033 Stream MySpace 1598 Download mp3.com.au 846 Download Triple J 1508 Stream ABC Count Type Source
Bit Rates For Files in Newcastle Survey Feb 2007
ID3 Tags Allows attributes to be embedded in sound files Possible attributes include: Artist name Song name Genre Filesize, bit rate, sample rate Important for players to identify songs Important for listeners managing mp3 libraries Search for “id3 tag editor” on the internet and you will find many freely available programs to manipulate these tags More information at http://www.id3.org/
ID3 Tags in the Local Music Survey 81% of songs did not have the “title” tag set 78% of songs did not have the “artist” tag set 85% of songs did not set the “genre” tag
ABC Music Awards Archive ABC Local Radio changed their website in March 2008 Deleted their 2500 page archive of previous music awards entries They have allowed me to republish the information
Referrals From Google

Maintaining a community based music website with open source software

  • 1.
    Maintaining a CommunityBased Music Website with Open Source Software Open Source Developers Conference 27 th November 2009
  • 2.
    In the beginning…Newcastle has a thriving music scene Members of the community lacked cohesion No support from commercial radio .com excitement in 1999
  • 3.
  • 4.
  • 5.
    Newcastle Music Directorywww.newcastlemusic.com Online since 1999 Over 4500 local artists listed Focuses on music from the Hunter Valley and Central Coast Site has over 8000 pages Updated frequently Sections include: News Links Videos Gig Guide
  • 6.
    TE Archive Onlinesince mid 2002 Site is located at 15min.org (ie. 15 minutes of fame) Features articles from the TE liftout from May 1998 to March 2006 Over 13,000 pages Over 6,500 images
  • 7.
    Geographical Focus Focusessolely on musical artists and websites from Newcastle and the surrounding regions Allows the site to be comprehensive
  • 8.
  • 9.
    Why “Local” Music?You can talk to the musicians after the gig
  • 10.
    Why “Local” Music?You can talk to the musicians after the gig The musicians will play weekly/monthly in your area
  • 11.
    Why “Local” Music?You can talk to the musicians after the gig The musicians will play weekly/monthly in your area CDs are cheaper
  • 12.
    Why “Local” Music?You can talk to the musicians after the gig The musicians will play weekly/monthly in your area CDs are cheaper Support the economy of your local region
  • 13.
    Why “Local” Music?You can talk to the musicians after the gig The musicians will play weekly/monthly in your area CDs are cheaper Support the economy of your local region Local musicians don’t mime
  • 14.
  • 15.
    Why Do ThisFor Free? From Wednesday’s keynote talk: Volunteers are “not motivated by financial or material gain” Volunteers are “motivated by ideology”
  • 16.
    Volunteering Many localmusos effectively volunteer their time for their passion Much of the local music industry is run on goodwill
  • 17.
    Aims Focus strictlyon the region around Newcastle, to the point of actively excluding news from other areas
  • 18.
    Aims Provide theNewcastle region with a musical identity of its own that bands and artists are proud to identify with.
  • 19.
    Aims Provide theregion with a comprehensive list of current gigs and events.
  • 20.
    Aims Provide acomprehensive list of links to locally based websites that relate to the music scene.
  • 21.
    Aims Provide up-to-dateand informative profiles for as many local bands and artists as possible.
  • 22.
    Aims Provide mapsand information about local pubs, clubs and musical venues.
  • 23.
    Aims Provide anarchive historical information about the local music scene for long term reference.
  • 24.
    Aims Provide aunified voice to highlight the musical talent of the region to people from other areas.
  • 25.
    Aims Meet andcollaborate with other people interested in the local music scene
  • 26.
    Looking Back …Started compiling links and information about the music scene in 1999 Have published over 20,000 pages about the local music industry on the internet Have volunteered over 7000 hours to the local music scene Received an award from in October 2005 for volunteer services to the local music industry Involvement in local music festivals and competitions All while working full time as an Engineer
  • 27.
  • 28.
  • 29.
    Effects of theRegional Focus Never any argument about what should be included on the site Restricted market – only really of relevance to the people of Newcastle (~1million people) Allows the website to become vwery comprehensive (~80% coverage?)
  • 30.
    Types of InformationBands and Artists Radio Larger Bands and Orchestras - Marching Bands, Orchestras, Choirs Services Music Shops
  • 31.
    Types of InformationRecording Studios Mobile Discos Music Organisations Record Labels Production Tuition
  • 32.
    Types of InformationMedia Events Managers and Agents Sound and Lighting Promotion Photography
  • 33.
    Sources of InformationFree webhosting services such as Geocities etc. Mp3.com Their own fully hosted domains mp3.com.au www.abc.net.au/newcastle as part of the ABC Newcastle Music Awards PureVolume.com MySpace.com Blogs on services such as blogger.com Press releases
  • 34.
    Sources of InformationNewspapers flickr.com Triple J Unearthed YouTube.com Booking agents and promoters websites Facebook Google Calendars Twitter ABC Contribute Word of mouth
  • 35.
    Link to ExternalWebsites Often the artist will maintain a bio/gig guide etc on another website newcastlemusic.com has no login system
  • 36.
    Automated Aggregation ofInformation More recently websites have allowed access to information via: RSS API Copying information from external websites (with the artists permission)
  • 37.
    Embedded YouTube VideoBandwidth supplied by YouTube Information retrieved from YouTube via RSS Playtime Author Upload date About 2000 videos are embedded in the site
  • 38.
    RSS Really SimpleSyndycation Many websites automatically generate RSS “feeds” when you update them Livejournal MySpace Blogspot Youtube Flickr etc. RSS allows people to subscribe to your blog Alerts people that you have updated your website without them visiting the site. Is a form of XML (eXtensible Markup Language)
  • 39.
    Artists Blog EntriesAutomatically brought into the site daily via RSS feeds Subscribed to over 3500 myspace blogs from the Newcastle music industry Provides an excellent source of up to date news about the local music scene
  • 40.
    The Gig GuidePublished around 1300 local events per month An additional 500 events from outside the region on the artists profiles Significant effort is made to filter out the local gigs
  • 41.
    Number of LocalGigs Published
  • 42.
  • 43.
    The Gig GuideDatabase Required a complete list of local venues and artists Compilation of such lists took over 5 years Required significant local knowledge Currently contains: 650 venues 4500 artists
  • 44.
    Venues from theNewcastle Region Indexed using Google Maps
  • 45.
  • 46.
  • 47.
    Sources of GigInformation Booking agents Venues Band Homepages Google Calendar Various other sources
  • 48.
    Maintaining the GigGuide Identifying and removing duplicate entries
  • 49.
    Maintaining the GigGuide Identifying and removing duplicate entries Catching misspelled band or venue names
  • 50.
    Maintaining the GigGuide Identifying and removing duplicate entries Catching misspelled band or venue names Deleting cancelled gigs from the database
  • 51.
    Maintaining the GigGuide Identifying and removing duplicate entries Catching misspelled band or venue names Deleting cancelled gigs from the database Ensuring that the correct band and venue are linked to a particular event in the database
  • 52.
    Maintaining the GigGuide Identifying and removing duplicate entries Catching misspelled band or venue names Deleting cancelled gigs from the database Ensuring that the correct band and venue are linked to a particular event in the database Specifying the time of an event if not otherwise stated.
  • 53.
    Updating Gig InformationIdentify source Disable all future gigs from that source For each gig: Re-enable if already found in database Add new event if not found This allows for cancellations/modifications
  • 54.
    Maintaining the GigGuide Database maintenance is not performed on the webserver Uploaded twice a week Much more stable and maintainable as a result
  • 55.
    Gig Archive Over50000 gigs from around the region All gigs link to artist profile and map of venue Archiving started in November 2006
  • 56.
    Open Information Localbooking agents and promoters are protective about their business practices Which venues they represent How often venues have entertainment Contact details for venues and artists
  • 57.
    Open Information Localbooking agents and promoters are protective about their business practices Which venues they represent How often venues have entertainment Contact details for venues and artists
  • 58.
  • 59.
    Open Source SoftwareAll websites built on open source software Apache MySQL Wordpress Various libraries (Magpie RSS etc etc)
  • 60.
    Open Source SoftwareAllowed the management these websites and the data behind them on a shoestring
  • 61.
    In for thelong haul … Open Source technologies allow flexibility and continuity
  • 62.
    The Stickybeak Inuse on all websites since 2002 Released in 2004 Operates in a similar way to Google Analytics Available from www.stickybeak.com
  • 63.
    Newcastle Music DirectoryPage Requests Per Day
  • 64.
  • 65.
  • 66.
  • 67.
  • 68.
    Web vs PrintMedia Internet and print media very different types of content Advertising in music print media often considered content on music websites Case Study: Reverb Street Press
  • 69.
    Reverb Street PressContent Street Press content was analysed during late 2007 Approximately 10% of the space was devoted to local artists Over 40% of the page space contained advertisements
  • 70.
  • 71.
  • 72.
  • 73.
    Conclusion Open sourcetechnologies have allowed the Newcastle Music Directory to grow in an adaptive and economical manner over a sustained period of development.
  • 74.
    Conclusion The resultis a website which provides relevant information about the music scene to the people of the Newcastle region.
  • 75.
  • 76.
  • 77.
    Local Music onthe Internet Library of over 13,000 songs Sourced from: JJJ Unearthed Mp3.com.au MySpace ABC 1233 Awards All are indexed by: File type Stream/download Stereo/mono Play Time File Size Sample Rate Bit Rate
  • 78.
    Newcastle Survey February2007 515 Download MySpace 2033 Stream MySpace 1598 Download mp3.com.au 846 Download Triple J 1508 Stream ABC Count Type Source
  • 79.
    Bit Rates ForFiles in Newcastle Survey Feb 2007
  • 80.
    ID3 Tags Allowsattributes to be embedded in sound files Possible attributes include: Artist name Song name Genre Filesize, bit rate, sample rate Important for players to identify songs Important for listeners managing mp3 libraries Search for “id3 tag editor” on the internet and you will find many freely available programs to manipulate these tags More information at http://www.id3.org/
  • 81.
    ID3 Tags inthe Local Music Survey 81% of songs did not have the “title” tag set 78% of songs did not have the “artist” tag set 85% of songs did not set the “genre” tag
  • 82.
    ABC Music AwardsArchive ABC Local Radio changed their website in March 2008 Deleted their 2500 page archive of previous music awards entries They have allowed me to republish the information
  • 83.