Wednesday, November 30, 2011

Project Social: Report 11

Hey!

A lot has been done in this project yet despite other deadlines in another course, but now when that's over and done with there will be even more social integration!

Last Week
I've been thinking of some ideas that ephracis thought would be a good idea for me to do since it's really connected to this project.

I will find a way to have Stoffi take control over the fileformats that it can take care of (like YouTube clips for example). It will be very useful when users are sharing links to songs or playlists over the Internet with each other. I'll design a startup window that lets you as the user change what you want Stoffi to take control over.

So far I've made a hastily done sketch of how that window can look like.




I know it's ugly with all the unused space and whatnot, but it is hastily done. But the end result will be awesome! :)

Challenges
The only challenge was how to design the window, but since it's pretty small and simple it wasn't much hard work.


This Week
Now I will focus on finding a way to edit the Windows registry so Stoffi can take control of its fileformats using C#.

So far I've gathered that I will have to use a namespace called "Microsoft.Win32". This will give us access to the various functions in order to change the registry.


Challenges
One challenge will be to alter the registry without causing any damage to Windows, but since one can easily find out how to access any fileformats in the registry it shouldn't be too mindboggling. Still, it doesn't hurt to tread carefully.


Bye for now.

Project Genesis: Report 11

The new beta is out and I have started to write parts of the final report. I am currently outlining the whole API and writing a short tutorial for it. I have also managed to bring in both the Download page and the Tour page into the new website.

Last week
Last week I finished off the website after the dreadful upgrade to Rails 3. It is finally back to and beyong where it was before the upgrade. I also managed to squeeze out a beta release of Zhou and some extra content to the new website.

I have started a little on writing the API docs and hope that I will be able to mix some report writing and studies for other courses now.

Challenges
There was really no big challenges last week. Everything went a lot smoother than expected. The upgrade packaging went extremely smooth. The only frustration was the loss of my Internet connection.


This week
I have not yet finished the control panel on the website and I really need to do that since the beta is actually released. But since it's server side I decided to do it after the release. It should take just a couple of hours anyway.

This week I will also do some heavy documentation, write a few tests and synchronize the Hackathon group.

Challenges
The biggest challenge is the challenge: to find time. I have a few other courses coming up with exams and it's time to wrap up the report soon as well. But I'll manage to get it all done. I'll just have to sleep less.

Tuesday, November 29, 2011

Beta update: cloud and social integration

Finally! I got my Internet connection back today.

I just upgraded beta to the latest Zhou code. Most of this upgrade comes from our Hackathon effort during the last few months.

If you are running Stoffi Beta then it should upgrade automatically, or you can just download the beta. Note that this is beta quality so there are a lot of bugs we haven't fixed yet and many translations are missing.

Firstly, it features integration with our new cloud platform, which has been born through Project Genesis. It allows you to synchronize all your applications and even remotely control them using another computer, phone or tablet.

Thanks to Project Social the new cloud platform is integrated with other social services. It is possible to login with Facebook, Twitter, Google, SoundCloud, Rdio, LinkedIn and Vimeo. It is also possible to share a song on both Twitter and Facebook. We are working on bringing in more external services and more integration.

I had some extra time to also implement some cool features outside the Hackathon scope. You can now generate a random playlist from an existing selection of tracks. Great for when you want to create a playlist to have in the car or on your portable player. I will try to bring additional features such as burning and synchronizing with devices in later releases.

Lastly, you can now tell Stoffi to pause playback while the computer is either locked or you have logged off.

If you want to try out our new Facebook integration you will need to get special testing permissions to our Facebook App. Since the whole platform isn't properly tested yet the app is still in development mode which means that you need special permissions to add it. So you need those permissions before you can login with Facebook.

Saturday, November 26, 2011

A peak at the new download page

The upcoming beta of Zhou is imminent. Zhou has been in feature freeze for a while and has since been polished and tested. It is now ready for packaging and deployment.

In the meantime I wanted to give you a small taste of what Zhou will bring. So here's a screenshot of our new download page:

New on left, old on right. Click for larger image.

Zhou will bring with it a pretty big upgrade to our website. It will now feature accounts and the ability to connect Stoffi with the website, all part of our hackathon this fall.

The new website will feature more soothing color theme and the colors are based on the Swedish flag, blue and yellow. The aim is to make it easier on the eyes. I'm starting to dislike that orange color in the current version.

So, what do you think? Love it? Hate it? Tell me in the comments.

Tuesday, November 22, 2011

Project Visual: Report 2

Second week. A bit more technical, but not much to say. Happy about the progress i have made though.

Last week
Initial implementation is coming along nicely, and the rough initial code design is holding so far. No big surprises or problems, and though i am yet to testrun the system for loading plugins, everything in place is compiling neatly just as suspected.

Challenges
None so far really, except for minor things i did not already know about visual studio 2010. Found a great tutorial on dynamic load in C# which is making progress pretty easy.

This week
More implementation, and the first testruns of the system. If evrything is working (i hopefully know in a day or two), the the rest of the week, and most of next week, is going to be about building the plugin interface.
The next report is likely to be way more interesting.


Challenges
Some synch will soon have to be made between myself and ephracis for the communication between the plugin system and the main application. Should prove to be easy enough, but still needs to be done.

Monday, November 21, 2011

Project Genesis: Report 10

Last week I aimed for having the OAuth consumer and provider code working again, as well as mobile and embedded views.

I manage to fix all that but the testing showed up some smaller problems that needed fixing.

Last week
I finished the OAuth code and thanks to we now running Rails 3 and the latest gems adding more services is just a few lines of code. I managed to add quite a few additonal services to our login system.

It's possible to login with:
  • Facebook
  • Twitter
  • Google
  • SoundCloud
  • Rdio
  • LinkedIn
  • Vimeo

I also got the mobile and embedded version back up. I was more lucky here. No modification was required - it still works the same.

Challenges
The biggest challenge last week was during the final testing where I found a severe bug in the new oauth gem. Due to a faulty parser the signignature would be corrupt if parameters included special characters. At the same time Rails uses a scheme where parameters are in the form object[attribute]=value. Of course this would prevent all post and put requests to fail on any object.

There was also a problem with the new oauth-plugin gem. Firstly it didn't remember that you'd already authorized an application and would ask you everytime you logged in to your account. There was a method to override but it was only called after the redirect, since the plugin required the request to be post. I corrected the code and got it working.

I plan on submitting patches as soon as the release is over and I get some more time to spare. Right now I am fully focused on getting the beta released.

This week
This takes us to what I'm going to do this week. I must fix the last bit of the website and then move on to doing a beta release. After the beta I will start to document the API and the part that will be in the report. If there's time I will start with filling the new website will all previous content such as news, about, contact, and so on. Which is pretty much just a copy-paste effort but I haven't prioritized it yet.

Challenges
The releases has been somewhat of a russian roulette for me. And even though the last ones have been very painfree, I think that the release will still be the biggest potential threat to screw some things up.

Other than that I don't see any threats right now. It's mostly just tedious work, but it's almost done. :)

Our 100th post

Woohoo!

This is our 100th post on the blog!


Stay tuned for a new beta in a few days with some really cool features such as synchronization, remote control, pause while locked, playlist generator and much more.

Thursday, November 17, 2011

Project Visual: Report 1

So, my first report is a couple of days late, but apart from that Im off to a pretty good start.

Have done some research into the tech we might need for the visualizer, and apart from that I've been giving a little thought to code design and...fighting the visual studio 2010 installer...

Last week
The interesting stuff first. My research covered two areas, namely what tech to use for the graphics themselves, and how dynamic load works in C#.
The original plan was to use openGL for the graphics, since it is a well supported and platform independent library. The only problem is that it did not have C# bindings, and solving that problem on our own is something i think we might do best to avoid if possible.
Looking for an easy solution for this, i found openTK, and open source C# library that wraps openGL, openAL and openCL. Just what the doctor ordered, and more.
So this seems to be our choice for now (until maby we find some big dealbreaking issue with it or until we find something better).

This bring me to the (rather short for now) point of code design. The idea is to not have the the graphics library tightly integrated directly into the plugin system, but rather to have a class in between so that we may more easily replace openTK should the need arise.

I have just touched the subject of dynamic load, but it seems really easy to do.

Challenges
Lastly i have been setting up the development environment. This meant getting a copy of visual studio 2010. After several failed attempts at installing (a perfectly legitimate copy) as well as some digging around in developer forums to find a solution to the problem it turned out i had to download a trial version and then unlock it with the version i had...fun times.

This week
Getting to know the Stoffi code and brushing up on my C# a bit. Reading up on how to do dynamic load a bit more. If i have time i will start doing some experimentation and I will at least get a little coding started (in fact, the first lines have already been written, though it doesn't do anything yet).

Challenges
The biggest hurdle for me is likely going to be C#, as it has been at least three years or so since i last used it. Not at all worried though, and i expect to be running at full speed ahead in no more than a week or two.

Tuesday, November 15, 2011

Project Genesis: Report 9

A huge bump in the road. I have been forced to recreate the whole web framework of the upcoming cloud service. I have been hard at work for the last two days and maybe I will be able to finish the work tomorrow.

Last week
Last week I managed to finish up everything on Stoffi in order to release the beta. The only thing left was to confirm everything and do some testing before release.

Then, as I needed to add translation to the website (since part of it is displayed inside Stoffi it needs to be displayed in the same language as Stoffi) I came to a dead end.

I have been using Rails 2.3 for a while now and Rails 3 have been out for quite some time. This time around the guys behind Rails have been hard at work migrating all existing documentation to Rails 3. It has been really hard from time to time to find documentation for Rails 2. On several occasions I've had to tweak example code and snippets in order to get it to work.

But setting up the routes for translation finally killed Rails 2 for me. There was no way I could get the routes setup properly so I went ahead and upgraded the app to Rails 3.

The upgrade was a disaster. I actually managed to get all of my code to work with Rails 3 but the recaptcha plugin threw some errors and the upgrade plugin provided by the Rails guys didn't upgrade all my files, so I got stuck with some bastard hybrid app that was neither 2 nor 3. I decided to create a whole new Rails 3 app and just move my code into it.

The biggest problem was that my faithful restful_authentication plugin didn't work with Rails 3. So I have moved to Devise. It is really slick and I got registration and login working yesterday. I then moved on to make sure that the translation routes worked properly and I got everything working late last night.

The translation system of Stoffi is actually pretty awesome.

Path
First, it looks for a locale in the path. So for example www.stoffiplayer.com/en/news will give you news in English and www.stoffiplayer.com/sv/news will give you news in Swedish.

Top domain
If no locale is found (like in www.stoffiplayer.com/news) then it will check the top level domain. This means that if we ever register stuff like stoffiplayer.se or stoffiplayer.ru, then the website will detect the language using the TLD.

Subdomain
If that fails (like when it's stoffiplayer.com) then it will look at the subdomains. This means that we could register stuff like se.stoffiplayer.com and ja.stoffiplayer.com, but since my SSL certificate only covers stoffiplayer.com and no subdomains I don't think we'll ever do this.

Browser setting
If no locale still haven't been detected then it will check the "Accept-language" parameter in the HTTP header sent from the browser. This means that if you set a language in your browser the website will detect it and display the proper language.

Location
If your browser doesn't send out any information about the language then Stoffi will do a lookup on your IP address (don't worry, it's not saved anywhere) and detect which country it originates from.

This week
Right now I am working on getting the OAuth client part working again so it's possible to connect to your Facebook, Twitter or Google account.

When this is done I'll move in the OAuth provider code so we can have Stoffi Music Player, Stoffi Remote and other third parties login and access the cloud services.

Lastly, I will get back the support for mobile and embedded views and the Juggernaut code so we can push out changes of objects to clients.

When all that is done I will verify that everything works, both on the website and in Stoffi, and also check with Gadd so the iOS app is still working. Then I'll do the beta release.

Challenges
I think that getting the OAuth provider stuff working in Rails 3 will be a challenge. The only instructions for the plugin are a blog post from the author and it dates back to 2007. The plugin may have been updated since then but we'll see how it works out. I haven't found any other plugin either.

I may also require some effort to get Juggernaut to play with our new setup but that plugin is very well maintained and will hopefully not offer too much trouble.

Since I am currently two weeks behind schedule on the beta release there may be a need to postpone the stable release of Zhou (we need to test it properly in beta before we release it into stable). But if I make that decision I'll make another post about it here on the blog.

Monday, November 14, 2011

Project Social: Report 8 & 9

Hello!

Alright! Two weeks have passed since my last report, and with nothing much to show for it. Why? Internet got broke.

Last Two Weeks
Ok, I say it like I had no Internet what so ever for the last two weeks. But it's actually been 5 days with little to no connection to the Internet. It started behaving badly on October 30:th and then got better 5 days later! But it was really slow, didn't even have 1 Mbps download speed then. So I had to wait a couple more days for it to get more stable and faster, but now I'm good!

The Problem
So what caused this to happen you might wonder. Well, I haven't really been doing my homework on it but I've heard from several sources that it's an mobile app that messed up the 3G net. And that app is Wordfeud!

There are probably people who can give a better answer about this than I can (seeing as I didn't even check to see if this even was so) but my guess is that Wordfeud had so many users which made the app send so many updates that it then broke the 3G net.

The Joy
So, anyways, now I have had Internet for a little more than a week. And boy is it wonderful! Now when the horrible times of having no Internet (at home) is over, I can work more on Stoffi. Or can I? It's not like I don't want to work on Stoffi and this project (really, I do), but I have to put my attention towards my other courses as well. And seeing as there are 3 of them, not including this project, my attention can get drawn to them.

But it's not all bad. I mean, there's not much left to do with this project. Well, except for the documenting of course. But other than that, what's left to do is to have Stoffi share a play-list to Facebook and also share songs to Google Buzz. So when thinking of what's left with this project, I'm not concerned.


That's it for now, see you next week.

Sunday, November 13, 2011

Password security at Stoffi

Our new and upcoming cloud service includes the ability to create user accounts. In today's climate security is a highly debated and important topic.

We have recently seen how Sony got hacked and Valve's Steam service was just compromised. In the former case vital information was not properly encrypted. Here in Sweden a popular blogging website called bloggtoppen.se had their database compromised a few weeks ago and they stored all their users passwords in clear text.

Do I need to tell you that here at Stoffi we take security seriously?


Encrypted communication
First of all I just bought us an SSL certificate which lets us offer a secure HTTPS connection. This provides both encryption and verification, so you know that any information sent is sent to us and that it is only readable by us. I will force the server to use a HTTPS connection at least during the login procedure.

Secure transmission
But I don't trust HTTPS completely. This year a CA server was compromised and there are known vulnerabilities in HTTPS and SSL. So that's why I have added an additional measure to enhance security during login and registration.

When you submit your password to our server (via login, registration or password reset) the passwords will be hashed using the SHA256 algorithm and salted with your email. If you look closely you will see that when you press "login" the password box changes as the password is hashed before it is sent to our server.


Secure storage
When your hashed password arrives at our server it is again hashed, this time using the SHA1 algoritm. Here we use a random salt along with a key stored in a configuration file on the server. This means that an attacker must get access to both the database and the server files in order to perform an offline attack.

Further, the hashing on the server side is digested 10 times which means that any offline attack will take 10 times longer.


Nothing is 100% secure
The worst and most unsecure thing is password reuse. If you use the same password everywhere then your whole online identity is only as secure as the worst website you have registered at. Make sure you use different passwords on different websites, or at least keep different passwords for the top most sensitive websites (for example Facebook, Google or PayPal).

Also remember that if an attacker gets access to your email account then he or she can just perform a password reset on any of your accounts, including Stoffi.

Stay safe!

Wednesday, November 9, 2011

Project Remote: Report 8


Last week
I designed and implemented a proposal of how the GUI could look. The app now also stores the authorization token in the keychain of the device, which saves the user from typing in his username and password every time the app starts. The various buttons and sliders also synchronize (animated) their state when the app receives an updated configuration from the server.

Challenges
Designing a GUI that feels like home in iOS without being a complete copy of the existing iPod app.

This Week
This week I will continue working mostly on the GUI. I will also implement the functionality of some existing interface elements, namely the skip-buttons (restart song, skip to next).

Monday, November 7, 2011

Project Genesis: Report 8

The week after the Shang release. It has been somewhat slower since I have divided my time between a new course, an old course and Stoffi, plus some relaxation to celebrate the new release.

But nevertheless, I have managed to get some new stuff going. Like a working registration flow on the website.


Last week
Rightly after I released Shang to stable I pushed some updates to Zhou and successfully merged it into the beta branch. This means that Zhou is now in feature freeze and work on Qin can now begin in trunk.

Furthermore I managed to find some time to polish the new website's login flow, integrating the new reverse turing test. I also stabilized the code in Stoffi that links/unlinks Stoffi with an account.

Challenges
The challenge this week as to fix the flow of the login/registration procedure on the website. Since we have two flows (a dialog and a complete website) and many calls are made using AJAX there has been a few complications making it take a lot longer than I expected.

This week
This week I'll verify the login/registration procedure on mobile devices and inside Stoffi. I'll also take a look at the problem surrounding communication from Stoffi to the server, where requests starts to timeout. If I manage to get this working I can start on releasing a beta upgrade and deliver these new features to beta testers.

Challenges
I bet that the communication problems will be a hard nut to crack. It could be that the requests lingers after they have been completed and are not cleaned up properly. I/O resources are unmanaged, I get the timeout on the third and following requests and the network queue has room for 2 requests. Sounds obvious enough but you never know.

Thursday, November 3, 2011

Update: no more punishment for missing Flash

I just dumped an update on the stable channel. It fixes a pretty serious bug that triggered when you ran Stoffi without having Flash installed for Internet Explorer. If this was the case you would get pretty screwed. You would enter an endless loop of warnings that you do not have Flash installed (no mention that it HAD to be for Internet Explorer, Firefox is not good enough apparently).

We never meant to punish you for not having Flash for Internet Explorer installed. Really sorry about that to those affected.

Background
For YouTube playback we use an embedded web browser. The browser in .NET runs Trident which is the core of Internet Explorer. As Google does not yet support embedded HTML5 players with a JavaScript exposed API, and Trident doesn't support HTML5 on YouTube, we are forced to use Flash.

The Problem
So Stoffi was supposed to stop and warn if no Flash was detected. However, this warning itself yelled "Stop!" to the media manager which at every "Stop!" tried to tell the YouTube player to stop. This caused a "no flash" error to occur again, and so the loop was created.

Unfortunately this happened when even as you tried to play a normal track not just a YouTube track (hence why the bug was so serious). A single warning (but no loop) was also triggered on startup which I found really annoying.

The Solution
First I made sure that the warning would only trigger directly after a Play action invoked by the user by double-clicking a track and I also made sure it would only occur once. After that I fixed the endless loop by putting in a number of checks around the media manager where it checks if Flash has been detected before it tries to talk to the YouTube player. Hopefully this will prevent similar or loops from happening.

Lastly, I modified the warning so it now explicitly mentions that it must be Flash for Internet Explorer. It also asks the user to download Flash and does so by opening up an instance of iexplore.exe and directs it to the website for getting Flash. My ambition would be to either move away from Flash fully and use HTML5 (which is hard since we need a new browser + support from Google) or bundle Flash with our installer (which may be hard because of licensing).

How To Upgrade
I just pushed out a new installer for those who doesn't have Stoffi already. For those of you who have Stoffi, it should upgrade automatically without any action needed to be taken on your part. If you have changed upgrade policy you may need take some action depending on the policy.

Special thanks to Thommy Siverman for finding and investigating this bug.

Wednesday, November 2, 2011

Project Mind Reader: Report 7

Trouble ahoy!

Last week
I started implementing my code on a much larger scale then before and actually integrated it with the rest of this project. Then after a while I decided to test the code I had written by primarily adding files to the library and then removing them again, after doing this several times the application crashed with a stack overflow error. I finally gave up and started commenting out the code that was not working to be able to push it. When pushing there were some merge conflicts and I decided to skip a lot of the changes all together, since they didn't seem to be working anyway.
So most of my work disappeared in a the hellish flame that is merge conflicts.


ChallengesFinding and removing the stack overflow error, that supposedly happens in infinite loops, though I am very confused where such could have come from.

This WeekI will try to get working code so I can start testing it but as you might have guessed, to implement Mind Reader in the current release is a long gone dream.

Project Social: Report 7

Hello!

Two days late for this week's post, but better late than never, right?

Last Week
For the last week I extended the share function to also include Twitter. So you can also post your favourite song to Twitter now as well as Facebook, at the same time and with only one click of a button! This will help you to tell your friends what your favorite music is, without any hassle.

Furthermore, we have a listen function that sends data to Facebook's ticker whenever you start to play a song. We're waiting though for Facebook to launch its ticker so that we can use it, but it's done otherwise. It works in such a way that all you have to do is stay connected to Facebook via Stoffi and it will send the data for you automatically!

Challenges
Testing to share to Twitter proved to be difficult as there was some trouble in connecting to Twitter in the first place. But that has been taken care of now and we can now share to Twitter perfectly!


This Week
Testing the share function so it works properly for Twitter, and that it continues to work great for Facebook as well. I'll also do some documenting on the side.


Be seeing you next week.

Tuesday, November 1, 2011

Project Remote: Report 7

The remote is finally functional!

Last week
I spent a lot of time trying to push my changes of the logged in users configuration to the server. With the help of Christoffer, the problem was fixed and the changes to the configuration made from the app can now be pushed to the server. This means the app can now remotely control the state of a desktop Stoff player!

Challenges
Solving the request authentication problem. It was very hard to debug as the problem could be on either the client or server side, or even both. We discovered that my OAuth library signed the request in a slightly different way than the server expected, which resulted in the server denying all requests except simple GET requests (which were used to retrieve the current configuration). This was fixed by packaging the request in a different way that allowed the library to sign it correctly.

This Week
This week I will make sure all the interface elements currently in the app are correctly hooked up to the configuration, so that all buttons and sliders actually affect the desktop player being controlled. I will also create a new design of the GUI, keeping in mind that the app should look native to iOS.

Project Genesis: Report 7

Oops! Forgot to write the report yesterday. Got caught up in all the exciting work. But here it is.


Last week
Last week I took some time off the Genesis Project so I could prepare and release our Shang update. It was really fun. I updated the website with an awesome new tour feature. I also managed to polish the code some extra before the release. I actually moved pretty much all of the startup code around. I managed to get the startup time down to just a forth of what it was before. Stoffi had actually started to see a longer and longer startup time during the development of Shang in the summer. This little update got rid of that and we have actually landed on a better startup time than Xia.

One big part of the startup is the scanner. So I also had to improve the scanner. It had also been getting slower and slower during the development of Shang and was about 10 times slower than Xia. But I moved some code around there as well and cut some updates. One really neat trick was to only update the progressbar every 100th scanned song. Since each update of the progressbar requires the seperate scanner thread to make a call to the main GUI thread and have it update the GUI the overhead was quite large. With these smaller tweaks I managed to make scanner faster than Xia.

Challenges
The biggest challenge was to get the upgrade mechanisms working. I successfully created a proper upgrade DLL for the settings and also got all the code working which was supposed to upgrade Xia to Shang. But as I was about to do a test on the stable channel (all upgrade tests are performed in the special test channel) I noticed that quite a few installations did not have the UAC prompt. Why this is I have no idea, but some people must have gotten them, since I actually got a feature request asking for its removal.

Without UAC Stoffi doesn't get administrator rights. Without those rights Stoffi can't upgrade itself.

But this is actually good. Since the Xia installations are in the protected Program Files folder and I won't be able to support them for very long (Zhou will be their last version, then they will all lose admin rights and won't be able to get Qin). So I urge everyone to reinstall Stoffi and I promise you that I will make sure that one of our most awesome features will shine fully the next upgrade.

This week
This week started with a merge of all current trunk code into the beta branch. This means that our Zhou version is now in beta - a place where we only apply polishes, bugfixes, performance improvements and smaller tweaks. No new features.

I did manage to sneak in two new features though. The first one is a really cool playlist generator which I am pretty fond of. It is a feature request from Olle Gällmo, whom is the "Stoffi Hackathon Overseer". With it you can generate a playlist by letting Stoffi select a number of random tracks from a given list of tracks. The second feature is the ability to see all similar YouTube tracks.

So what I will do this week is to polish up the code, make it stable. I will also merge in our experimental reverse turing test into the registration procedure.

I hope that I will be able to release a beta (and an alpha) update this week as well.

Challenges
The main challenge will probably be to get the network code more stable and predictable. I have a few tricks up my sleave so we'll see if I will manage to get everything going really good before the end of the weekend.