Here are two examples where a cloud computer may come in handy:

1) If you have ever tried to host services on a local machine through a bread-and-butter Internet Service Provider, you will be delighted to be allocated a persistent IP-address. You can also decide on the ports you want to be accessible or shut bypassing ISP-policy, e.g. blocking outbound port 80 (default HTTP-port).

2) If you have ever used a shared host for hosting websites, you may have been met with software that doesn’t work. This can be because your scripts, plug-ins or other software are using languages not supported by the host. For example many general purpose hosts are not providing support for Python and Ruby. Another problem is libraries and other dependencies. If you want to have access  to more advanced photo-editing through e.g. Imagemagick then your fate is in the hand of the provider and what stack/software/libraries/dependencies they wish to provide. You are also stuck with using FTP-client as many shared hosting providers do not grant you SSH access to their servers.

Signing up for AWS access

Setup is fairly easy. First you need to go to the AWS website and sign in or sign up with an Amazon account. As part of the registration Amazon will validate your phone number by calling you with an automatic voice service to give you a verification code, so keep your phone with you as you start the registration process. Follow through the web form for registration a Amazon account, then a

Creating the EC2 Instance

AWS has many services, but for this blog post we stick to the EC2. Click on the image to enlarge it.

Once registered and validated you should have access to the Management Console. If not you find this by first entering the AWS frontpage, then on the top of the window selecting “My account/Console” then “AWS Management Console”. This should bring up a new view with all the services that AWS provide (don’t be dazzled, there is very much information, but luckily one can survive just using a couple of them). Select the EC2 option in the orange section named “Compute & Networking”. This should bring up another view. Before continuing, go to the top right corner of the screen (to the right for your name) and select the region that is closest to you. (More on regions)

The EC2 view. From here you can create and administer your EC2 instances, and services related to these.

In the EC2-view you can easily set up and administer your instances. One thing to be aware of it that once the instance is created some settings persists with it, and these can only be changed by creating a new instance. This can cause you many hours of extra work if you do many changes to your instance, and it could also make it impossible to access the device (if you loose your encryption keys and passwords.)

There are three things we need to pay attention to at this point: what kind of instance do we want, how should we create the authentication, and how do we deal with persistant storage.

Choosing an AMI

AMI is the abbreviation for Amazon Machine Image. These are the images with the virtual appliances which are used for instanceiate the EC2 (which is basically a virtual machine). You can choose between free public AMIs, commercial images where you have to pay or you can even create an image yourself (the last is out of scope for this post). For this tutorial we are going to use a commonly used image – The Amazon Linux AMI.

From the EC2-view click the new instance button, then choose a classical view for selecting an AMI. As you can see there are several options within Linux and Windows. The ones marked with a star got a free tier if used with the micro instance, so no need to worry about the costs yet.

Creating the keys

To connect to the EC2 you will need SSH-keys which can identify you. There are two ways of creating these keys, either create the key locally and share the public key with the EC2 instance, or you can let Amazon create the keys and send you the public key for the instance. I recommend creating these locally as you then have both the private and the public keys. There seems to be a limit on downloading the keys from Amazon, and this may cause problems if you loose the key. It is also an advantage to have both keys in case you use two or more computers to communicate with your EC2 instance.

To create an SSH-key open your terminal and enter: ssh-keygen -t rsa -C “your@email.example” A tutorial is also provided by GitHub.

Upload this SSH-key when you assign keys in the EC2 setup.

Getting an IP-address

When your instance is up and running you get an access point address where you can find and connect to your instance. I would however recommend that you get an IP-address to this point. Not only is it easier and shorter, it also gives you a way of abstract your virtual machine from the address. Amazon provides you with an Elastic IP, and with this you can change which EC2-instance the IP address should point to. This makes it easy to e.g. start up a new and more powerful instance and then quickly change the IP address to this instance instead of letting the user endure the longer time a virtual reboot would take. At some point you would like to map a DNS address to the Elastic IP so e.g. sub.domain.com or domain.com points to your instance, at this point it is nice to just have to configure the elastic IP and not have to change the DNS record when you want to change instances.

Test your instance

The instance should now be up and running. Now try to connect to your instance using SSH. The login username may change from AMI to AMI, if your AMI is ubuntu the username is usually ubuntu, if you use the standard Amazon Linux AMI the username is ec2-user. 

ssh -v ec2-user@123.123.123.12'

Changing the security settings

If you are not able to login it may be that you need to open the firewall to your instance. As you created your instance, you assigned the instance a security settings group.

If you have any problems connecting to a service using a specific port, you should check here to ensure that the traffic is not being blocked at the firewall. For SSH you will need to open port 22.

Creating an alias for easier access

This is not a part of the setup process, but it’s a neat little trick for making access easier. Instad of remembering the whole address in the terminal when you use SSH to connect to your instance, make an alias in the .bashrc or .bash_profile.

If you connect writing ssh -i $HOME/keys/aws.pub 1233.4556.7789.1 then put the string below into your .bash_profile and source this.

alias my_ec2_connect='ssh -v -i $HOME/keys/aws.pub 1233.4556.7789.1'

(Disclaimer to point two: lovholm.net is in fact hosted on a shared host by one.com – WordPress is very convenient to host at providers such as one, since it’s built on ubiquitous web-stack PHP+MySQL, and shared hosts are generally affordable and even easier to set-up than dedicated. One does also have a good admin dashboard from which you also can do some simple rerouting – more on this in a later post)

The image is from Flickr and is provided under a CC-licence. The image is associated to cloud computing and IBM, not Amazon. 

Thank you for reading!

Ok, I’m very enthusiastic about readers, so after a day with many readers in Google Analytics I may be like this, especially with many revisiting users. I hope to see you again:

SVG – Scalable Vector Graphics – er, som gitt av navnet, et skalerbart vektorbasert grafikk-markup. Vektorgrafikk er bedre egnet til skalering enn raster, da bildet ikke består av små bildeelementer (Pixel – kort for nettopp picture element), men formler for opptegning av grafikken. Nå har det seg riktignok slik at ikke alt fungerer like godt i begge grafikkverdener. Siden vektorer er basert på forskjellige punkter på et lerret og forbindelsen mellom disse, enten igjennom punkter, rette linjer, eller former for interpolerte kurver er vektorer bedre egnet til å representere abstrakte bilder og modeller, mens raster med sine pixelpunkter er godt egnet for å representere sensordata fra for eksempel bildebrikker. Det at fonter i dag representeres som vektorer gjør at disse kan skaleres uten tap i kvalitet (forsøk selv ved å forstørre denne teksten til det 10-dobbelte).

Forskjell på vektor og bitmap (raster) font.

For å overføre rasterdata til vektordata og motsatt må bildet “parses” eller konverteres, dette er ikke alltid like enkelt da bilder med mange detaljer og mye informasjon kan være vanskelig å tolke for programvaren. Har du et bilde av en rett svart strek på hvit bakgrunn er denne enkel å prosessere, men har du derimot et hus er det værre.

Et rasterbilde av et hus forsøkt vektorisert, uten hell.

SVG-formatet

En SVG formatert fil kan åpnes i flere forskjellige programmer, med ganske forskjellig visning. Dersom du åpner en SVG-fil i en leser som støtter formatet og prosesserer dette til grafikk vil du få følgende visning av kartet med norske fylker

Dersom du åpner samme fil i en teksteditor (jeg bruker textmate på mac, men du kan benytte TextWrangler som er gratis på Mac, og NotePad++ for Windows. Benytter du Linux har du helt sikkert også en favoritteditor) vil du se en struktur som i utganspunktet ser ganske uoversiktelig ut, men som ganske enkelt kan forståes. SVG-standarden har en rekke forskjellige funksjoner, men jeg kommer kun til å nevne noen i denne artikkelen.

I filene du kan benytte for å tegne kommuner og fylker på kartet er det i hovedsak paths eller stier som benyttes. Stien for å tegne Oslo kommune ser slik ut:

  <path
     id="oslo"
     class="land oslo county03 ostlandet"
     d="M 136.54875,466.88777 C 133.12579,463.23916 133.1611,459.6972 136.62044,459.6972 C 138.64571,459.6972 141.17041,464.85896 140.28004,467.17924 C 139.54285,469.10032 138.55185,469.02291 136.54875,466.88777 z"
     style="fill:#c0c0c0;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-opacity:1"
  />

Verdt å merke seg her er hvordan d-taggen i elementet inneholder en ganske lang liste. Denne listen består av instruksjoner som benyttes når nettleseren, eller andre programmer tolker SVG-filen om til en visuell representasjon. M er move to. Her begynner tegneprosessen. C er kurve. Hit sendes tre punkter som videre benyttes for å interpolere en kurve. Dette fortsetter fram til formen sluttes med kommandoen Z. For å markere et fylke benytter vi fill-argumentet i style-taggen. Som du kan se består denne av en grå fargeverdi nå (elementert kjære Watson, både R – to første etter hashtag, G – to i midten – og B – to til høyre – har samme hexadecimale verdi. RGB)

Norske fylker og kommuner

Ok, nå har vi noe kunnskap om SVG, men hvordan er det vi får markert enten fylker eller kommuner  med fine farger? Jo, det neste vi trenger er SVG filer med Norgeskart.

Wikipedia har en fin SVG-fil over norske kommuner. Høyreklikk på bildet og last ned fila.

Wikipedia har også en fin SVG-fil over norske fylker. Høyreklikk på bildet og last ned fila.

Finn noen fine farger. Dersom du ikke har noen favoritt RGB-hex fargekoder anbefaler jeg Adobes gode Kuler verktøy.

Editer SVG-fila

Filene du nå har lastet ned inneholder path elementer for enten fylker eller kommuner. Ta fila for det politiske nivået du ønsker, åpne denne filen i din favoritteditor, søk deretter på navnet på kommunen eller fylket og skriv inn en annen farge for kommunen eller fylket du ønsker å endre. For eksempel kan du ta utgangspunkt i tekstutsnittet av Oslo kommune fra ovenfor (Oslo er både fylke og kommune, og vil derfor være i begge filene. Merk: Dette er Oslo fra kommunefilen) og legg inn en annen fargekode for fyll.

  <path
     id="oslo"
     class="land oslo county03 ostlandet"
     d="M 136.54875,466.88777 C 133.12579,463.23916 133.1611,459.6972 136.62044,459.6972 C 138.64571,459.6972 141.17041,464.85896 140.28004,467.17924 C 139.54285,469.10032 138.55185,469.02291 136.54875,466.88777 z"
     style="fill:#FF0000;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-opacity:1"
  />

Du vil nå ha et rødt Oslo.

Et svart Hedmark, grønt Akershus og rødt Finnmark.

Et grønt Rendalen (Norges største kommune utenfor Finnmark) og Hamar.

Bilder:

Bilde for å illustrere forskjellen på vektor og raster font, samt bilde av dårlig vektorisert hus er lånt fra.

http://www.1stwebdesigner.com/design/pixel-vector-graphics-difference/

http://giorgioscappaticcio.wordpress.com/category/task-1-image-sourcing/

I have been using these services for about half a year now, and I am convinced these services as well as competing equals (e.g. Microsoft Azure) is a good tool that for everyone who want to experiment with web technology or create web businesses. This is not a new thing, cloud computing have been around for years, and many web services are running on Amazon services today, but through looking into them myself I understood how I can benefit from them in my work with the web and through the Internet, and new experiments

S3

Amazon Simple Storage Service provide affordable web space. In this specific market similar services have emerged, which also are easier accessible e.g. Dropbox – one of the best things since bread came sliced. The good thing about S3 is the tight integration with the other Amazon services, it is also good for hosting static web pages (pages that only needs to be served, without initial computation). Some FTP clients also include S3 support so you can use the S3 service as you would regularly use FTP for storage of files. Another advantage is that you can choose the S3 bucket (the entity in which you place your files on S3) to be distributed to all of Amazons data centres (for an additional cost). This is good if you need to transfer files to end user fast as there are less potential for bottlenecks using a local datacenter (by local here I mean the same continent.)

EC2

Amazon Elastic Compute Cloud is the real game changer. While most of the other services are specialised the EC2 is the general purpose machine. An EC2 instance is a virtual computer located in the cloud where you pay for the computing hours you use, a standardised unit being approximately: “CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.” Since the payment is by the hour, and the units are scalable it can be very flexible to your needs. The service enables you to make your hosting scalable both for scaling up, and scaling out.

Scaling up: The instances comes in several packages. From the modest and almost free micro instance to extra-large distributed platforms. You can change the processor performance and dedicated RAM on your instance, and changing the instance type just require a reboot and you can transfer your persistance storage onto the new virtual machine.

Scaling out: If you create an image with all your server variables, binaries, libraries and other stuff you need, or a method for loading this from a remote computer, you can in just a couple of minutes initialise new instances with these data, and since you pay by the hour you can e.g. have four computers running at times with high load and one during times with low load. The coolest feature is that you can measure trafic and load new servers as the service is running at lower performance than wanted.

Elastic-IP: 

The URL to the instances are normally quite arcane and difficult to remember. That is why this is quite good. Elastic-IP is just bridge between a IPv4 address that you get allocated, and the instance. Above I described how you could reboot an instance, but the downside to this is that while the instance is restarting your server would be down. To avoid this, you could create a clone of your instance #1 with the higher capacity settings, and when this is loaded, direct the IP to the new instance #2 and then turn off the instance #1. (Well, I guess you would loose the users’ sessions)

Do you want to set up an EC2 instance? (More about how)

RDS

RDS, or the Relational Database Service. You need this if you want to handle persistent data in a relational manner (and if you’re not hacking user-data into self-composed formatted files, or using a noSQL solution) and think life is to short to set-up and configure a SQL-database on the general purpose EC2 service.

When you create a new DB instance you can easily choose type. If you use Oracle you can bring your own licence or rent licence by a hour-licence.

For the RDS you can choose between (ex-Swedish, now Oracle) MySQL, Microsoft SQL Server or  Oracle (also Oracle *dough* ). The setup is very easy, and you can choose the size of the database, the backup retention period and other variables then start the instance. Check the security settings that the ports are open and that the DB can access traffic from your IP-address. Download SQL Workbench (MySQL) or SQL Developer (Oracle) (Sorry, not sure about good tool for MS SQL Server), insert the endpoint URL, username and password and get started.

Other services

The AWS selection is huge, but luckily you won’t need to know every service available to get started. (Click images for full size)

This picture show the AWS panel from which you administer your services, and as you can see there are several more services to explore. Many of these are specialised to e.g handle MapReduce or deploying specific web-stacks easily, but with the S3, RDS and EC2 it should be easy to get started.

The script worked, and found and returned about 80% of the artists, and around 80% of these again had the conceptually similar artist. What if the alteration could be done in the document itself? For the CSV based script the artists had to be extracted from the database, then parsed and then put into the document. With this attempt we can skip the middle step, and let the script run simultaneous with data being available to view for the users.

If you haven’t already used Google Docs, you should consider it, as it is a very convenient way of working with documents, especially where there are more users. Through Google Docs you can collaborate on working on documents, and since they are stored in clouds all users will instantly have the latest version. Another advantage is that whey you are working on the document simultaneously each user is editing the same document, so merging different versions is not a problem.

A great tool for combining Google Spreadsheets with the Python programming environment is the gspread module. After installing this package you only need to import the code and then you can with very few lines of code retrieve and update information in the spreadsheet cells.

The specifics for doing this task is pretty much these few lines. Import the package. Login, find correct document and do whatever you need to. (this excerpt won’t work. Check the whole script under – NB: mind the indentation. It may not be correct displayed in the browser.)

import gspread

G_USERNAME = 'your@gmail.com'
G_PASSWORD = 'yourPassword'
G_IDENTIFIER = 'document_identifier_checkdocument_url_in_your_browser'

# Connecting to the data source
gc = gspread.login(G_USERNAME,G_PASSWORD)
sht1 = gc.open_by_key(G_IDENTIFIER)
worksheet = sht1.get_worksheet(0)

for a in range(2,5124):
try:
name = worksheet.acell(get_val[a]).value
res = getCountry(name)
if res:
print res
country, last_id, name = res
worksheet.update_acell(set_name[a], name)
worksheet.update_acell(set_country[a], country)
except Exception as e:
print e

Above is the lines related to connecting Python to the google docs, under you can see the whole script and how the method I mentioned in a post earlier is used in this setting.

#!/usr/bin/python
# -*- coding:utf-8 -*-

"""
Clouds & Concerts - 2012
Ola Loevholm

Initialized from the commandline. Runs through The Google doc spreadsheet with topp 5000 artists, and
runs the parsing query against the Last.FM browser then enters the country and search string (for validation)
into the google docs.

"""

G_USERNAME = 'your@gmail.com'
G_PASSWORD = 'yourPassword'
G_IDENTIFIER = 'document_identifier_checkdocument_url_in_your_browser'

import sys, urllib, string, csv, time
import xml.etree.ElementTree as ET
import gspread

# Loads a dictionary with ISO 3166-1 abbreviations and codes
COUNTRIES = {"AF":"AFGHANISTAN","AX":"ÅLAND ISLANDS","AL":"ALBANIA","DZ":"ALGERIA","AS":"AMERICAN SAMOA","AD":"ANDORRA","AO":"ANGOLA","AI":"ANGUILLA","AQ":"ANTARCTICA","AG":"ANTIGUA AND BARBUDA","AR":"ARGENTINA","AM":"ARMENIA","AW":"ARUBA","AU":"AUSTRALIA","AT":"AUSTRIA","AZ":"AZERBAIJAN","BS":"BAHAMAS","BH":"BAHRAIN","BD":"BANGLADESH","BB":"BARBADOS","BY":"BELARUS","BE":"BELGIUM","BZ":"BELIZE","BJ":"BENIN","BM":"BERMUDA","BT":"BHUTAN","BO":"BOLIVIA, PLURINATIONAL STATE OF","BQ":"BONAIRE, SINT EUSTATIUS AND SABA","BA":"BOSNIA AND HERZEGOVINA","BW":"BOTSWANA","BV":"BOUVET ISLAND","BR":"BRAZIL","IO":"BRITISH INDIAN OCEAN TERRITORY","BN":"BRUNEI DARUSSALAM","BG":"BULGARIA","BF":"BURKINA FASO","BI":"BURUNDI","KH":"CAMBODIA","CM":"CAMEROON","CA":"CANADA","CV":"CAPE VERDE","KY":"CAYMAN ISLANDS","CF":"CENTRAL AFRICAN REPUBLIC","TD":"CHAD","CL":"CHILE","CN":"CHINA","CX":"CHRISTMAS ISLAND",
"CC":"COCOS (KEELING) ISLANDS","CO":"COLOMBIA","KM":"COMOROS","CG":"CONGO","CD":"CONGO, THE DEMOCRATIC REPUBLIC OF THE","CK":"COOK ISLANDS","CR":"COSTA RICA","CI":"CÔTE D'IVOIRE","HR":"CROATIA","CU":"CUBA","CW":"CURAÇAO","CY":"CYPRUS","CZ":"CZECH REPUBLIC","DK":"DENMARK","DJ":"DJIBOUTI","DM":"DOMINICA","DO":"DOMINICAN REPUBLIC","EC":"ECUADOR","EG":"EGYPT","SV":"EL SALVADOR","GQ":"EQUATORIAL GUINEA","ER":"ERITREA","EE":"ESTONIA","ET":"ETHIOPIA","FK":"FALKLAND ISLANDS (MALVINAS)","FO":"FAROE ISLANDS","FJ":"FIJI","FI":"FINLAND","FR":"FRANCE","GF":"FRENCH GUIANA","PF":"FRENCH POLYNESIA","TF":"FRENCH SOUTHERN TERRITORIES","GA":"GABON","GM":"GAMBIA","GE":"GEORGIA","DE":"GERMANY","GH":"GHANA","GI":"GIBRALTAR","GR":"GREECE","GL":"GREENLAND","GD":"GRENADA","GP":"GUADELOUPE","GU":"GUAM","GT":"GUATEMALA","GG":"GUERNSEY","GN":"GUINEA","GW":"GUINEA-BISSAU","GY":"GUYANA","HT":"HAITI","HM":"HEARD ISLAND AND MCDONALD ISLANDS",
"VA":"HOLY SEE (VATICAN CITY STATE)","HN":"HONDURAS","HK":"HONG KONG","HU":"HUNGARY","IS":"ICELAND","IN":"INDIA","ID":"INDONESIA","IR":"IRAN, ISLAMIC REPUBLIC OF","IQ":"IRAQ","IE":"IRELAND","IM":"ISLE OF MAN","IL":"ISRAEL","IT":"ITALY","JM":"JAMAICA","JP":"JAPAN","JE":"JERSEY","JO":"JORDAN","KZ":"KAZAKHSTAN","KE":"KENYA","KI":"KIRIBATI","KP":"KOREA, DEMOCRATIC PEOPLE'S REPUBLIC OF","KR":"KOREA, REPUBLIC OF","KW":"KUWAIT","KG":"KYRGYZSTAN","LA":"LAO PEOPLE'S DEMOCRATIC REPUBLIC","LV":"LATVIA","LB":"LEBANON","LS":"LESOTHO","LR":"LIBERIA","LY":"LIBYA","LI":"LIECHTENSTEIN","LT":"LITHUANIA","LU":"LUXEMBOURG","MO":"MACAO","MK":"MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF","MG":"MADAGASCAR","MW":"MALAWI","MY":"MALAYSIA","MV":"MALDIVES","ML":"MALI","MT":"MALTA","MH":"MARSHALL ISLANDS","MQ":"MARTINIQUE","MR":"MAURITANIA","MU":"MAURITIUS","YT":"MAYOTTE","MX":"MEXICO","FM":"MICRONESIA, FEDERATED STATES OF",
"MD":"MOLDOVA, REPUBLIC OF","MC":"MONACO","MN":"MONGOLIA","ME":"MONTENEGRO","MS":"MONTSERRAT","MA":"MOROCCO","MZ":"MOZAMBIQUE","MM":"MYANMAR","NA":"NAMIBIA","NR":"NAURU","NP":"NEPAL","NL":"NETHERLANDS","NC":"NEW CALEDONIA","NZ":"NEW ZEALAND","NI":"NICARAGUA","NE":"NIGER","NG":"NIGERIA","NU":"NIUE","NF":"NORFOLK ISLAND","MP":"NORTHERN MARIANA ISLANDS","NO":"NORWAY","OM":"OMAN","PK":"PAKISTAN","PW":"PALAU","PS":"PALESTINIAN TERRITORY, OCCUPIED","PA":"PANAMA","PG":"PAPUA NEW GUINEA","PY":"PARAGUAY","PE":"PERU","PH":"PHILIPPINES","PN":"PITCAIRN","PL":"POLAND","PT":"PORTUGAL","PR":"PUERTO RICO","QA":"QATAR","RE":"RÉUNION","RO":"ROMANIA","RU":"RUSSIAN FEDERATION","RW":"RWANDA","BL":"SAINT BARTHÉLEMY","SH":"SAINT HELENA, ASCENSION AND TRISTAN DA CUNHA","KN":"SAINT KITTS AND NEVIS","LC":"SAINT LUCIA","MF":"SAINT MARTIN (FRENCH PART)","PM":"SAINT PIERRE AND MIQUELON","VC":"SAINT VINCENT AND THE GRENADINES",
"WS":"SAMOA","SM":"SAN MARINO","ST":"SAO TOME AND PRINCIPE","SA":"SAUDI ARABIA","SN":"SENEGAL","RS":"SERBIA","SC":"SEYCHELLES","SL":"SIERRA LEONE","SG":"SINGAPORE","SX":"SINT MAARTEN (DUTCH PART)","SK":"SLOVAKIA","SI":"SLOVENIA","SB":"SOLOMON ISLANDS","SO":"SOMALIA","ZA":"SOUTH AFRICA","GS":"SOUTH GEORGIA AND THE SOUTH SANDWICH ISLANDS","SS":"SOUTH SUDAN","ES":"SPAIN","LK":"SRI LANKA","SD":"SUDAN","SR":"SURINAME","SJ":"SVALBARD AND JAN MAYEN","SZ":"SWAZILAND","SE":"SWEDEN","CH":"SWITZERLAND","SY":"SYRIAN ARAB REPUBLIC","TW":"TAIWAN, PROVINCE OF CHINA","TJ":"TAJIKISTAN","TZ":"TANZANIA, UNITED REPUBLIC OF","TH":"THAILAND","TL":"TIMOR-LESTE","TG":"TOGO","TK":"TOKELAU","TO":"TONGA","TT":"TRINIDAD AND TOBAGO","TN":"TUNISIA","TR":"TURKEY","TM":"TURKMENISTAN","TC":"TURKS AND CAICOS ISLANDS","TV":"TUVALU","UG":"UGANDA","UA":"UKRAINE","AE":"UNITED ARAB EMIRATES","GB":"UNITED KINGDOM","US":"UNITED STATES",
"UM":"UNITED STATES MINOR OUTLYING ISLANDS","UY":"URUGUAY","UZ":"UZBEKISTAN","VU":"VANUATU","VE":"VENEZUELA, BOLIVARIAN REPUBLIC OF","VN":"VIET NAM","VG":"VIRGIN ISLANDS, BRITISH","VI":"VIRGIN ISLANDS, U.S.","WF":"WALLIS AND FUTUNA","EH":"WESTERN SAHARA","YE":"YEMEN","ZM":"ZAMBIA","ZW":"ZIMBABWE"}

 

# Connecting to the data source
gc = gc = gspread.login(G_USERNAME,G_PASSWORD)
sht1 = gc.open_by_key(G_IDENTIFIER)
worksheet = sht1.get_worksheet(0)
# Iterates through XML-structure and removes the namespace, for easier navigation in getCountry()s ElementTree.findall()
def remove_namespace(doc, namespace):
"""Remove namespace in the passed document in place."""
ns = u'{%s}' % namespace
nsl = len(ns)
for elem in doc.getiterator():
if elem.tag.startswith(ns):
elem.tag = elem.tag[nsl:]
# getCountry - where the magic happens. Encodes string with artistname to url, then query musicbrainz search engine.
# parses the XML-answer and get the name, id and country of the first returned element (with highest weight)
# returns country name i a) artist is found through the search engine b) artist has a country associated to the profile, otherwise returns False
def getCountry(name):
name = urllib.quote_plus(name)
BASE_URL = "http://musicbrainz.org/ws/2/artist/?query=%s&format=xml&method=advanced" % (name)
print "Querying: %s" % (BASE_URL)
try:
search_input = urllib.urlopen(BASE_URL)
# Checks whether HTTP Request Code is 200 - if not goes to sleep for 5 seconds // Inded for 503 Code
http_code = search_input.code
if http_code != 200:
# print "Could not access: %s \t Got HTTP Code: %s. 5 second cool-down" % (name, http_code)
time.sleep(5)
getCountry(name)
except Exception:
print "GETTING_ERROR: Something went wrong while getting HTTP"
return False
#search_xml = search_input.read()
#print search_xml
try:
tree = ET.parse(search_input)
remove_namespace(tree, u'http://musicbrainz.org/ns/mmd-2.0#')
feed = tree.getroot()
elem = feed.findall("./artist-list/")
#print elem[0].find('name').text
#print elem[0].get('id')
except Exception:
print "PARSE_ERROR: Something went wrong while parsing HTTP"
return False
try:
if elem[0].find('country') != None:
# print COUNTRIES[elem[0].find('country').text]
try:
country = COUNTRIES[elem[0].find('country').text]
except Exception:
print "Could not find key in countrylist error"
return False
return [country,elem[0].get('id'),elem[0].find('name').text]
else:
print elem[0].find('name').text + " has not any country associated\n"
return False
except IndexError, ValueError:
print "ERROR - COULD NOT GET DATA FROM %s\n" % (name)
return False

#Running through the values
get_val = []
set_name = []
set_country = []
for a in range(2,5124):
v = "B%s" % a
sn = "H%s" % a
sc = "G%s" % a
get_val.append(v)
set_name.append(sn)
set_country.append(sc)

for a in range(2,5124):
try:
name = worksheet.acell(get_val[a]).value
res = getCountry(name)
if res:
print res
country, last_id, name = res
worksheet.update_acell(set_name[a], name)
worksheet.update_acell(set_country[a], country)
except Exception as e:
print e

Her er en liten oppskrift på hvordan du kan gjøre dette på facebook sider du følger. Det er ikke alltid at social feed plukker opp disse, og dessuten er det fint å kunne lagre oppdateringer for ettertiden.

Det første du trenger er denne URL strengen:

https://www.facebook.com/feeds/page.php?format=atom10&id=241453889260137

Denne ganske ordinære URL strengen som parses som to argumenter av webserveren [:format =>  "atom", :id => "241453889260137"] (ruby hash syntax), et voila, du får en atom formatert RSS side tilbake. Det du gjør her er å bytte ut id fra URLen ovenfor med iden fra siden du ønsker å følge.

Men hva om det ikke er noe ID i side URLen på Facebook? Som f.eks i bildet under (tatt fra siden til Løvholm Digitale Medier – mitt forholdsvis nye enkeltmannsforetak).

Her blir det litt mer komplisert.Dersom du forsøker å laste inn atom RSS-feeden fra dette navnet vil du kun få en side med en feilmelding.

Vår vei til målet går igjennom fantastiske Open Graph, teknologien Facebook bruker for å organisere sine tjenester og sin verden, og dens API.

http://graph.facebook.com/lovdigmed

Her kan du bruke navnet som oppslag og få tilbake en JSON-formatert tekststreng med det du trenger å vite

Nå trenger du kun å gå tilbake til den første URLen vi hadde, og benytte IDen til siden (merk: ikke cover_id) for å få tilgang til RSS-feeden.

Nå kan du lese Facebook sideoppdateringer i din yndlingsleser sammen med alt annet interessant du abonnerer på

As you may see from the current state, not everything is up to date, but I hope you have patience as I will now start to customise the theme (maybe even a post on WP PHP programming will follow).

For future updates, stay tuned – or subscribe to the RSS feed.

Hvordan fungerer en wordle?

Ingen er muligens bedre kvalifisert til å svare på dette enn Jonathan Feinberg, skaperen av Wordle, som på den sosiale programmingsspørsmålsiden StackOverflow presenterer framgangsmåten.

Først telles alle ordene, og sorteres slik at ordene med høyest frekvens kommer først. Deretter fordeles ordene utover lerretet slik at frekvensen gjenspeiles i størrelsen på ordene. De mest gjentatte ordene er de som dukker opp høyest. I løpet av telle-perioden må også ord som av grammatiske (eller andre) grunner ikke skal med fjernes. Ord som ‘er’, ‘som’, ‘i’, ‘på’ brukes ofte i norsk, men de har ikke en semantisk mening på enkeltordsnivå som kan reflekteres i en wordle. (ja, det er en distinksjon mellom å være f.eks ‘i Drammen’ eller på ‘på Drammen’, men denne reflekteres ikke når alle ord splittes). Når ordene og deres frekvens er bestemt begynner plasseringsjobben, og denne er ganske vanskelig å få til (forsøk  å lage en algoritme for å best organisere alle koffertene for bagasjerommet i en bil, hvor alle koffertene har forskjellige størrelse og fasong).

Hvordan gjør vi det?

Vi skal velge en løsning som ligger i mellom Wordle og en hvor vi må utvikle alt helt fra begynnelsen. Biblioteket WordCram for Processing gjør jobben ganske enkel. På WordCram sine sider kan du også se et bilde av den amerikanske grunnloven, og dersom du syntes vårt resultat blir likt så stemmer dette, siden det er fra denne visualiseringen inspirasjonen til vår versjon av den norske grunnloven kommer fra.

US Consitution visualized through WordCram

Vi kommer til å låne fire ting fra denne presentasjonen. Den enkleste å kopiere er bakgrunnsfargen som med sin offwhite minner om offisielle papirer (og da snakker vi ikke om 80 g/m² A4 papir). Denne fargen representeres ved RGB-hex fargekoden #F5F1E3.

Vi skal også låne begge fontene som brukes i visualiseringen. Den ene, med håndskriftassosiasjoner heter, typisk nok, Jane Austen og kan lastes ned fra dafont.com. Den trykk-inspirerte fonten heter AveriaSerif-Regular. Vi kommer til å bruke Gruesa versjonen av denne. Begge to kan lastes ned fra i pakken med Averia fonter fra IOtic.

Det siste, og kanskje største tingen vi kommer til å låne er framgangsmåten. Koden som brukes ser slik ut:

import wordcram.*;
import wordcram.text.*;

void setup() {
size(600,500);
background(#F5F1E3);

PFont averia = loadFont("Averia-Gruesa-48.vlw");
PFont janeAusten = loadFont("JaneAusten-48.vlw");

new WordCram(this)
.fromTextFile("grunnloven1814.txt")
.withFonts(averia, janeAusten)
.withColors(#000000)
.angledAt(0)
.withWordPadding(1)
.drawAll();

}

Vi har gjort noen endringer. Spesielt i forhold til font. Istedet for å laste inn systemfonter har jeg gått inn i Processings font verktøy (“Tools”->”Create font”) og laget .vlw filer for fontene.

Grunnloven

Vi har nå satt opp rammeverket hvor magien skal skje, men vi trenger fortsatt hovedingrediensen, nemlig Grunnloven av 1814. Den finner vi heldigvis på Wikisource (et godt sted for kilder). Kopier teksten, legg den inn i en tekstfil som igjen lagres i mappen data i Processing-sketchen din. Bruk en teksteditor som TextMate eller Notepad, ikke en tekstbehandler som OpenOffice eller Word.

Et voila

Ordsky Norges Grunnlov

Verdt å merke seg er at det kommer et nytt resultat fra den bakenforliggende algoritmen hver gang den kjøres. Derfor vil resultatet også kunne se slik ut

Wordle, eller Wordcloud, av Grunnloven 1814 i svart. Denne gangen med Jane Austen-foten mer dominant

In Technicolor

La oss arbeide videre med utgangspunktet vi allerede har. I visualiseringen av den amerikanske grunnloven brukte de blått og rødt. Dersom vi skal visualisere grunnloven slik at den fremstår “mer norsk” er den fargekombinasjonen som vi først tenker på (si gjerne ifra hvis du tenkte på en annen fargekombinasjon): rødt, hvitt og blått, eller som vi også kan kalle de: #EF2B2D, #FFFFFF, #002868.

Igjen fra Wikipedia:

The flag of Norway consists of a  dark blue  cross (colour: RGB #002868; Pantone PMS-281U) with white borders on a  bright red  background (RGB #EF2B2D; Pantone PMS-032U). The vertical arm of the cross is shifted towards the flag pole as in the other Scandinavian flags. The overall width−to−height proportions of the flag are 22 to 16, and the individual coloured fields measure 6+1+2+1+12 in width and 6+1+2+1+6 in height.

La oss endre koden fra tidligere til å ta med fargene:

import wordcram.*;
import wordcram.text.*;

void setup() {
size(600,500);
background(#F5F1E3);

PFont averia = loadFont("Averia-Gruesa-48.vlw");
PFont janeAusten = loadFont("JaneAusten-48.vlw");

new WordCram(this)
.fromTextFile("grunnloven1814.txt")
.withFonts(averia, janeAusten)
.withColors(#EF2B2D, #FFFFFF, #002868)
.angledAt(0)
.withWordPadding(1)
.drawAll();

}

Resultatet er nå annerledes og i farger (selv om hvitt på off-white blir noe rart. Hvilken farge mener du passer best?)

Nå, to år før grunnlovsjubileet har du en wordcloudversjon av grunnloven.

Artist and Country

One of the tasks at hand was to find out which country an artist came from, and whether they came from Norway or not. One way of doing this is to manually go through each artist and use preexisting knowledge about music to determine their country of origin, if stuck, use online services (aren’t we all mostly using Google as initial source of wisdom). Another alternative is to use online services first and then to use human preexisting knowledge to quality assure the final result.

On the Internet, vast amount of sources can be found. However, if you want to get the data without too much fitting, testing, and nitty gritty adaptation for every source, you have an advantage if there is as consistently structured repository you can tap from. Luckily, Metabrainz foundation has a large repository of musical meta-data known as Musicbrainz.

Below you find a script which should (partially) solve our problem by combining the data from Musicbrainz with data exported from our research data. That being said, this script is more a method than a finished product. It should be very easy adaptable, but it is an advantage if you know Python and handling CSV-files. Codeacademy has a good introduction to Python.

The core idea of the script is to take input with name and number of streams and turn them into output with name from the original datasource, number of streams from the original datasource, as well as country of origin, Musicbrainz-ID, and name parsed by Musicbrainz search engine (for initial quality assurance).

To make things simple there is only one successful output and that is if the name sent to the Musicbrainz search engine return an answer, and if that answer has a country associated with it. Be advised (that is also why I have marked the title with ‘try’), that the search engine may not return a similar result. Of that reason we also print the name of the artist we find so this later can be juxtaposed with the original name in the Excel spreadsheet (you are going to transform the CSV to Excel before reviewing aren’t you. Good tool is Google Refine). Another problem is that popular cultural phenomena, common nouns and tribute bands (probably in that order, descending) have same name. This is why a human is always needed, or semantic absolute URI associated with each phenomena. This leads me on to the last step before the code.

Other ways this could have been solved (let me know if you solve the problem in any of these ways)

The semantic way:

The data found in the Musicbrainz database is made available through a SPARQL endpoint named LinkedBrainz.If you know the right ontologies and is comfortable with performing triplestore queries, this is perhaps the most innovative and new-thinking way to solve the problem.

The Virtual Machine Postgres way:

Instead of doing a query on the server, you can be a gentleman and download the server onto your own machine. If you have VirtualBox (if you don’t have it, download it for free) you can run the server locally. An image file with the complete Musicbrainz database can be found on their webpages.

The code:

Here is the code used to solve this task. It can also be cloned from the Cloud & Concerts GitHub-page

#!/usr/bin/python
# -*- coding:utf-8 -*-

"""
Clouds & Concerts - 2012
Ola Loevholm

Called from command line:
The script reads a file named "topp1000_artister.csv" consisting of a list of artists and then tries to find out which country each artist comes from based on the name.
The name is given in the second column of the CSV file.

Called as a module:
The method getCountry() takes an artist name and checks this with musicbrainz seach engine. Returns the country if a) artist is found through the search engine b) artist has a country associated to the profile

"""

import sys, urllib, string, csv, time
import xml.etree.ElementTree as ET

# Loads a dictionary with ISO 3166-1 abbreviations and countries
COUNTRIES = {"AF":"AFGHANISTAN","AX":"ÅLAND ISLANDS","AL":"ALBANIA","DZ":"ALGERIA","AS":"AMERICAN SAMOA","AD":"ANDORRA","AO":"ANGOLA","AI":"ANGUILLA","AQ":"ANTARCTICA","AG":"ANTIGUA AND BARBUDA","AR":"ARGENTINA","AM":"ARMENIA","AW":"ARUBA","AU":"AUSTRALIA","AT":"AUSTRIA","AZ":"AZERBAIJAN","BS":"BAHAMAS","BH":"BAHRAIN","BD":"BANGLADESH","BB":"BARBADOS","BY":"BELARUS","BE":"BELGIUM","BZ":"BELIZE","BJ":"BENIN","BM":"BERMUDA","BT":"BHUTAN","BO":"BOLIVIA, PLURINATIONAL STATE OF","BQ":"BONAIRE, SINT EUSTATIUS AND SABA","BA":"BOSNIA AND HERZEGOVINA","BW":"BOTSWANA","BV":"BOUVET ISLAND","BR":"BRAZIL","IO":"BRITISH INDIAN OCEAN TERRITORY","BN":"BRUNEI DARUSSALAM","BG":"BULGARIA","BF":"BURKINA FASO","BI":"BURUNDI","KH":"CAMBODIA","CM":"CAMEROON","CA":"CANADA","CV":"CAPE VERDE","KY":"CAYMAN ISLANDS","CF":"CENTRAL AFRICAN REPUBLIC","TD":"CHAD","CL":"CHILE","CN":"CHINA","CX":"CHRISTMAS ISLAND",
"CC":"COCOS (KEELING) ISLANDS","CO":"COLOMBIA","KM":"COMOROS","CG":"CONGO","CD":"CONGO, THE DEMOCRATIC REPUBLIC OF THE","CK":"COOK ISLANDS","CR":"COSTA RICA","CI":"CÔTE D'IVOIRE","HR":"CROATIA","CU":"CUBA","CW":"CURAÇAO","CY":"CYPRUS","CZ":"CZECH REPUBLIC","DK":"DENMARK","DJ":"DJIBOUTI","DM":"DOMINICA","DO":"DOMINICAN REPUBLIC","EC":"ECUADOR","EG":"EGYPT","SV":"EL SALVADOR","GQ":"EQUATORIAL GUINEA","ER":"ERITREA","EE":"ESTONIA","ET":"ETHIOPIA","FK":"FALKLAND ISLANDS (MALVINAS)","FO":"FAROE ISLANDS","FJ":"FIJI","FI":"FINLAND","FR":"FRANCE","GF":"FRENCH GUIANA","PF":"FRENCH POLYNESIA","TF":"FRENCH SOUTHERN TERRITORIES","GA":"GABON","GM":"GAMBIA","GE":"GEORGIA","DE":"GERMANY","GH":"GHANA","GI":"GIBRALTAR","GR":"GREECE","GL":"GREENLAND","GD":"GRENADA","GP":"GUADELOUPE","GU":"GUAM","GT":"GUATEMALA","GG":"GUERNSEY","GN":"GUINEA","GW":"GUINEA-BISSAU","GY":"GUYANA","HT":"HAITI","HM":"HEARD ISLAND AND MCDONALD ISLANDS",
"VA":"HOLY SEE (VATICAN CITY STATE)","HN":"HONDURAS","HK":"HONG KONG","HU":"HUNGARY","IS":"ICELAND","IN":"INDIA","ID":"INDONESIA","IR":"IRAN, ISLAMIC REPUBLIC OF","IQ":"IRAQ","IE":"IRELAND","IM":"ISLE OF MAN","IL":"ISRAEL","IT":"ITALY","JM":"JAMAICA","JP":"JAPAN","JE":"JERSEY","JO":"JORDAN","KZ":"KAZAKHSTAN","KE":"KENYA","KI":"KIRIBATI","KP":"KOREA, DEMOCRATIC PEOPLE'S REPUBLIC OF","KR":"KOREA, REPUBLIC OF","KW":"KUWAIT","KG":"KYRGYZSTAN","LA":"LAO PEOPLE'S DEMOCRATIC REPUBLIC","LV":"LATVIA","LB":"LEBANON","LS":"LESOTHO","LR":"LIBERIA","LY":"LIBYA","LI":"LIECHTENSTEIN","LT":"LITHUANIA","LU":"LUXEMBOURG","MO":"MACAO","MK":"MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF","MG":"MADAGASCAR","MW":"MALAWI","MY":"MALAYSIA","MV":"MALDIVES","ML":"MALI","MT":"MALTA","MH":"MARSHALL ISLANDS","MQ":"MARTINIQUE","MR":"MAURITANIA","MU":"MAURITIUS","YT":"MAYOTTE","MX":"MEXICO","FM":"MICRONESIA, FEDERATED STATES OF",
"MD":"MOLDOVA, REPUBLIC OF","MC":"MONACO","MN":"MONGOLIA","ME":"MONTENEGRO","MS":"MONTSERRAT","MA":"MOROCCO","MZ":"MOZAMBIQUE","MM":"MYANMAR","NA":"NAMIBIA","NR":"NAURU","NP":"NEPAL","NL":"NETHERLANDS","NC":"NEW CALEDONIA","NZ":"NEW ZEALAND","NI":"NICARAGUA","NE":"NIGER","NG":"NIGERIA","NU":"NIUE","NF":"NORFOLK ISLAND","MP":"NORTHERN MARIANA ISLANDS","NO":"NORWAY","OM":"OMAN","PK":"PAKISTAN","PW":"PALAU","PS":"PALESTINIAN TERRITORY, OCCUPIED","PA":"PANAMA","PG":"PAPUA NEW GUINEA","PY":"PARAGUAY","PE":"PERU","PH":"PHILIPPINES","PN":"PITCAIRN","PL":"POLAND","PT":"PORTUGAL","PR":"PUERTO RICO","QA":"QATAR","RE":"RÉUNION","RO":"ROMANIA","RU":"RUSSIAN FEDERATION","RW":"RWANDA","BL":"SAINT BARTHÉLEMY","SH":"SAINT HELENA, ASCENSION AND TRISTAN DA CUNHA","KN":"SAINT KITTS AND NEVIS","LC":"SAINT LUCIA","MF":"SAINT MARTIN (FRENCH PART)","PM":"SAINT PIERRE AND MIQUELON","VC":"SAINT VINCENT AND THE GRENADINES",
"WS":"SAMOA","SM":"SAN MARINO","ST":"SAO TOME AND PRINCIPE","SA":"SAUDI ARABIA","SN":"SENEGAL","RS":"SERBIA","SC":"SEYCHELLES","SL":"SIERRA LEONE","SG":"SINGAPORE","SX":"SINT MAARTEN (DUTCH PART)","SK":"SLOVAKIA","SI":"SLOVENIA","SB":"SOLOMON ISLANDS","SO":"SOMALIA","ZA":"SOUTH AFRICA","GS":"SOUTH GEORGIA AND THE SOUTH SANDWICH ISLANDS","SS":"SOUTH SUDAN","ES":"SPAIN","LK":"SRI LANKA","SD":"SUDAN","SR":"SURINAME","SJ":"SVALBARD AND JAN MAYEN","SZ":"SWAZILAND","SE":"SWEDEN","CH":"SWITZERLAND","SY":"SYRIAN ARAB REPUBLIC","TW":"TAIWAN, PROVINCE OF CHINA","TJ":"TAJIKISTAN","TZ":"TANZANIA, UNITED REPUBLIC OF","TH":"THAILAND","TL":"TIMOR-LESTE","TG":"TOGO","TK":"TOKELAU","TO":"TONGA","TT":"TRINIDAD AND TOBAGO","TN":"TUNISIA","TR":"TURKEY","TM":"TURKMENISTAN","TC":"TURKS AND CAICOS ISLANDS","TV":"TUVALU","UG":"UGANDA","UA":"UKRAINE","AE":"UNITED ARAB EMIRATES","GB":"UNITED KINGDOM","US":"UNITED STATES",
"UM":"UNITED STATES MINOR OUTLYING ISLANDS","UY":"URUGUAY","UZ":"UZBEKISTAN","VU":"VANUATU","VE":"VENEZUELA, BOLIVARIAN REPUBLIC OF","VN":"VIET NAM","VG":"VIRGIN ISLANDS, BRITISH","VI":"VIRGIN ISLANDS, U.S.","WF":"WALLIS AND FUTUNA","EH":"WESTERN SAHARA","YE":"YEMEN","ZM":"ZAMBIA","ZW":"ZIMBABWE"}

# Iterates through XML-structure and removes the namespace, for easier navigation in getCountry()s ElementTree.findall()
def remove_namespace(doc, namespace):
	"""Remove namespace in the passed document in place."""
	ns = u'{%s}' % namespace
	nsl = len(ns)
	for elem in doc.getiterator():
		if elem.tag.startswith(ns):
			elem.tag = elem.tag[nsl:]

# getCountry - where the magic happens. Encodes string with artistname to url, then query musicbrainz search engine.
# parses the XML-answer and get the name, id and country of the first returned element (with highest weight)
# returns country name i a) artist is found through the search engine b) artist has a country associated to the profile, otherwise returns False
def getCountry(name):
	name = urllib.quote_plus(name)
	BASE_URL = "http://musicbrainz.org/ws/2/artist/?query=%s&format=xml&method=advanced" % (name)
	print "Querying: %s" % (BASE_URL)
	try:
		search_input = urllib.urlopen(BASE_URL)
		# Checks whether HTTP Request Code is 200 - if not goes to sleep for 5 seconds // Inded for 503 Code
		http_code = search_input.code
		if http_code != 200:
#			print "Could not access: %s \t Got HTTP Code: %s. 5 second cool-down" % (name, http_code)
			time.sleep(5)
			getCountry(name)
	except Exception:
		print "GETTING_ERROR: Something went wrong while getting HTTP"
		return False
	#search_xml = search_input.read()
	#print search_xml
	try:
		tree = ET.parse(search_input)
		remove_namespace(tree, u'http://musicbrainz.org/ns/mmd-2.0#')
		feed = tree.getroot()
		elem = feed.findall("./artist-list/")
		#print elem[0].find('name').text
		#print elem[0].get('id')
	except Exception:
		print "PARSE_ERROR: Something went wrong while parsing HTTP"
		return False
	try:
		if elem[0].find('country') != None:
#			print COUNTRIES[elem[0].find('country').text]
			try:
				country = COUNTRIES[elem[0].find('country').text]
			except Exception:
				print "Could not find key in countrylist error"
				return False
			return [country,elem[0].get('id'),elem[0].find('name').text]
		else:
			print elem[0].find('name').text + " has not any country associated\n"
			return False
	except IndexError, ValueError:
		print "ERROR - COULD NOT GET DATA FROM %s\n" % (name)
		return False

# If method is called from terminal. Iterates through topp1000 artists contained in a CSV-file in same directory.
if __name__ == "__main__":
	#name = sys.argv[1]
	csvfile = open("topp1000_artister.csv")
	outfile = open("topp1000_output.csv","w")
	artistlist = csv.reader(csvfile, delimiter=',', quotechar='"')
	for line in artistlist:
		result = getCountry(line[1])
		try:
			if result != False:
				result_string = "%s,%s,%s,%s,%s,%s\n" % (line[0],line[1],line[2],result[0],result[1],result[2])
#				print result_string
			else:
				result_string = "%s,%s,%s,%s\n" % (line[0],line[1],line[2],"No Country Found or fail occured")
#				print result_string
		except IndexError, ValueError:
			print e
			result_string = "Error on element: %s\n" % line[1]
		try:
			outfile.write(result_string)
		except:
			print "Write error happened with %s" % line[1]

And as always, I am most greatful for feedback! Hope this may come in handy!

In this real word example a Python scripts runs through the folder in which the script is located, resizes each image of the type png and jpg (hopefully the filenames corresponds to the format), when the images have been resized they are put inside a new folder.

Note that PIL ensures a correct aspect ratio and that the size variable just sets the boundary. In the code bellow the important feature is to have images that not exceeds 150px in width (standard convention with width and height). Passing the PIL Image object the size will transform the object into a thumbnail. The method by which the transformation should be performed can be added as an optional parameter. From the documentation: “The filter argument can be one of NEAREST, BILINEAR, BICUBIC, or ANTIALIAS (best quality). If omitted, it defaults to NEAREST.”

#!/usr/bin/python
# -*- coding: utf-8 -*-
import Image
import os
#Imports the Image and os libraries. os is part of standard libraries. Image is part of Python Image Library (PIL)
#PIL can be downloaded from: http://www.pythonware.com/products/pil/

#Creates sets and creates the directory onto where you want your files to be saved
outdir = "150images/"
os.mkdir("./"+outdir)
#If the directory already exists this will cause an OSError.

size = 150, 400 #Set the size that you want to resize your image to.
#Thumbnail automatically checks for ratio consistency so alter the important variable (height or width)

for files in os.listdir("."):
    #Sets the appropriate suffix to your files.
    outfile = os.path.splitext(files)[0] + "_thumbnail.jpg"
    #Transforms JPG formatted files
    if files.endswith(".jpg"):
        im = Image.open(files)
        try:
            im.thumbnail(size, Image.ANTIALIAS)
            im.save(outdir+outfile, "JPEG")
            print "Saved the file: %s" % outdir+outfile
        except IOError:
            print "cannot create thumbnail for '%s'" % infile
    #Transforms PNG formatted files
    if files.endswith(".png"):
        im = Image.open(files)
        try:
            im.thumbnail(size, Image.ANTIALIAS)
            im.save(outdir+outfile, "JPEG")
            print "Saved the file: %s" % outfile
        except IOError:
            print "cannot create thumbnail for '%s'" % infile

The script will execute once, but as you execute twice you will get an OSError, this is due to the folder being created in the beginning of the script. I have chosen to name this folder after the significant size boundary in the script (maximum 150px width). PIL is not a part of the standard library, but can easily be installed through pip or easy_install. Python is available on almost every platform and comes preinstalled on the Mac.