Tuesday, January 28, 2014

Status update: rewritten core, automated tests, and build script

Hi, all!

As you may have noticed, it has been quiet from me for a while now. There are several reasons for this. First of all I've been busy finishing up my master's thesis and starting my new job. My daughter has also started kindergarten which means she gets sick a lot more often now.

But another reason why I've been quiet is because I've been busy restructuring the development cycle of Stoffi. Starting with the next release I will finally have a more streamlined process which means that Stoffi can be updated and improved at a quicker rate.

In the new release cycle I will have a completely finished beta version where all the new and shine features are done, all that's left is to test it and make sure it's stable. While the testing is being done, I will be working exclusively on the alpha version where new stuff gets released early and often. From time to time I will collect all bug reports from the alpha, fix them, and release the fixes on both beta and alpha. Finally, when the beta is really stable, I will push that into the stable channel, the alpha will go into the beta channel, and a new alpha will be started.

That's the plan. But before I can get there I need to move everything into sync. So now I have actually been working on two new versions at the same time, so to speak. The beta has some new features, which have mostly been tested and are very close to being deemed stable enough, while I have also worked on the alpha which has even more features, and is almost ready to be moved into beta when the beta is released into stable. Still following me?

So right now everything is aligned and almost ready. I have a few more tests to make before I am completely confident that the beta code is ready to be moved into the stable channel.

But this is not all. I have also been busy streamlining the release process around Stoffi. I have created a build script which automates a lot of the tasks that had to be done manually before whenever Stoffi was to be released. Such as updating the version number, compiling the two installers in Visual Studio, creating an upgrade package, and so on. My goal is to have a build server in the future and have this script run every night, pushing out all changes to alpha testers. But for this to work I need to both have a dedicated Windows machine with Visual Studio installed, and learn the script to build the tool for settings migration whenever the way settings are stored is changed. So there's some work left. But the current build script will at least save me from all the manual work involved in releasing Stoffi.

I have also started to write tests, both unit tests and integration tests, for Stoffi Core and the Stoffi website. This will help me keep Stoffi stable now when Stoffi is growing into becoming a full ecosystem with a cloud service, music player, and hopefully a smartphone app in the future.

So while you may not have heard much from me lately, I can guarantee you that you'll have a lot to look forward to. Two new versions of Stoffi (one almost fully tested and ready to be released soon), a faster bug hunt, and a more streamlined release process.

Also, now when I have finished school and started work, I have no labs or exams. When I get home, I'm free.

So that's it for now. I still have a few more tests to run before I can release a new version: playlist sync, file associations, and integration with Facebook, Twitter, and Last.fm. But when all that is tested and no bugs are found, I will officially start this new era of Stoffi.

Have a great day!

Thursday, November 7, 2013

Moving to SQLite for storing application settings

So it's been a while since the latest updates. The biggest reason is because I have been busy with a lot of other stuff lately but it is also due to what it is that I've been working on regarding Stoffi.

If you've followed Stoffi on Facebook or Twitter you'll know that I've been working on a new version for OS X. While doing this new version I started to get myself stuck in some serious crashes. After a lot of investigation I found out that the reason behind the crashes was how I was storing application settings in Stoffi.

Since the start of Stoffi, settings have been stored in XML format using the ApplicationSettings which is built into .NET. This has some downsides which have been discussed in various bug reports. The biggest issue is that reading and writing is extremely slow. Somewhat related is that whenever the settings are updated the whole application will freeze for a second, which is why I decided to save the settings only when Stoffi is closed. The result is that Stoffi needs to spend a lot of time reading and writing settings at startup and shutdown.

So for a while there has been some discussions about moving to another backend for storing settings. Both SQLite and some form of NoSQL database has been suggested. However, this move would mean a lot of work and could break a lot of stuff.

When I got stuck in crashes while working on the OS X version however, I was more or less forced to fix this long standing issue. So for the last months I have been slowly working on rewriting the whole part of Stoffi which saves every setting of Stoffi. At the same time I have been juggling 133% study pace and helping out with Haylie.

But today I can finally say that the new system is working and is pretty stable. I have verified that it works on both OS X and on Windows. There's some spit and polish left but most of it is there.

So what are the benefits? Well they are there but you probably won't notice them too much. Stoffi will now have a slightly faster startup and shutdown which is nice. Settings are also now saved as soon as you change something. So if your computer crashes or Stoffi is for some other reason not properly shutdown, you will get right back to where you were the next time you start it. This is pretty sweet for those of us who are in the habit of abusing Stoffi pretty heavily by crashing it and shutting it down in the most merciless ways.

A downside is that you can't just open the settings file and edit it in notepad anymore. Now you need to download the SQLite client and have some knowledge of SQL if you want to mess around with the settings. But if I know my users well this won't be much of an issue for most of them.

So, with this large rewrite finally done I can go back to some other cool work. I plan on implementing playlist management in the OS X version before I move on to releasing the pending upgrade to the Windows version. If you want to help speed up the process head over and download the beta version of Stoffi and report any issues you find.

Wednesday, June 19, 2013

Beta update: DI.fm playlist import, faster shutdown, improved UTF support

In a few hours I will fly to Hong Kong. So I'll be gone for a few weeks, but before I leave I want to give you a new and fresh beta to tear apart.

This update improves the playlist importer. You can now import playlist URLs from Digitally Imported (di.fm) without Stoffi crashing. There's also tons of updates regarding the playlist synchronization and listen statistics. Stoffi can now handle songs, artists and playlists containing special characters. This required a bit of work on both the server and client but hopefully it's working alright now. Let me know if you run into any problems.

The shutdown is now also faster, or at least it should feel like it, as the "Saving settings..." dialog is now gone. Stoffi will still save the settings and .NET's built in settings engine is still really slow, but now all that is done after the window disappears. This should make it appear as if Stoffi is shutting down in less than a second.

I also managed to find and fix a bug that caused Stoffi to crash while searching on YouTube. This crash didn't happen every time so it was a bit hard to pin it down but it's gone now.

Take the beta for a spin and report any bugs you may find. As far as I know all bugs are now fixed so unless anyone finds anything more I will release this to all users.

Take care!

Monday, June 17, 2013

Alpha update: more views, Jamendo, YouTube playlists

Time for another update to the alpha version of Stoffi.

The biggest of the new features in this update is the additional view mode. You can now select to view your songs as icons, content tiles, a simple list, or the old detailed list. It works just like in Windows Explorer except we don't support arbitrary icon sizes (yet). Hopefully this will make it a bit easier to browse your music.

Speaking of music, I've added more. This time the source is Jamendo. I haven't had time to really dig into Jamendo myself yet but it seems to be a bit different than what's available on SoundCloud. Also, the meta data seems to be much better at Jamendo.

The last big new feature is the ability to import YouTube playlists. Right now you can do this by selecting to add a playlist from YouTube in the context menu and then pasting the URL into the text box. Later I will do some magical connection between your YouTube account and Stoffi so your playlists can be automatically imported into Stoffi and synchronized back to YouTube.

As usual, there's also all the latest bug fixes coming in from the beta version of Stoffi as well. But you still shouldn't expect any stability in the alpha. Most of these new features are really untested and may crash a lot. But if you want to see the direction I'm taking Stoffi in for the next upcoming releases you can download the alpha and try it out. If you want to help out testing new stuff and send bug reports you should focus on the beta, though.

Have a great day!

Wednesday, June 5, 2013

In the meantime, at Stoffi HQ...

Summer!

Today I took the last exam for the semester and when I got home I finished and submitted my last report.

So now I have almost three months before I start on my master thesis in the fall. This means that I can get some time to wrap things up with the new Stoffi version and hopefully release it. Yay!

However, I have a planned trip to Hong Kong with my family between June 11th and July 19th. So there may not be anything happening on the Stoffi front until after I get home. But don't worry! I have something for you while you wait. I found some cash a couple of months ago and one of the things I spent it on was my office/diaper room (this is were all the magic happens).

So I took some picture to show you the awesome results.


Here's some bad guys after running into Luke and Chewbacca.


But Luke and Chewie are backed up by a hiding Han Solo.


An overview of the stand off. Note Princess Leia coming in to rescue the boys with a sneak attack from behind.

This is really the place where I just put all the excess characters I got from all the other models. I may actually build some environment and incorporate the cables from the screens into it to make it look really cool. It would also make the characters less prone to falling over.


Here's Leia showing some of the tactical plans over at the rebel headquarters on the planet Hoth.


A lone rebel running for his X-wing.


Some imperial troopers going in for an attack on the rebel base. Also, a drone!


A TIE bomber, perhaps running away from that big bad-ass X-wing in the background?


The complete space scene with an A-wing, a B-wing, a Tantive IV and the planets Alderaan and Endoor. Also, an astroid field (which actually looks like a single planet, guess I have to paint the whole room black so it blends in more).


There's Yoda! He's in the middle of a fight with some droidekas. You may not see it clearly here but one droideka is actually destroyed. Two more standing. Yoda also has a AT-RT behind him and a clone trooper coming in from the left.


Luke is hanging from a snow speeder over the imperial troops.


Finally, the whole of my little HQ. There's still some stuff I didn't show (like ranging lightsaber duel between Darth Vader and old Obi-Wan Kenobi) but you can spot the Millennium Falcon in this picture which is my (so far) most expensive piece (except the desk with adjustable height).

Don't worry, there will be more. I plan on building both a scene from Tatooine and one from Endor later on.

Wednesday, May 1, 2013

Companies on why they limit passwords (hint: because they suck)

I just came across a post on Ars Technica where they asked companies why they limit their users' passwords (in length and/or allowed characters), making the passwords less secure. The responses where pretty much the same from everyone and one commenter really captured the essence of the different responses. Schpyder wrote:
Schwab: "No comment."
MS: "Not a concern. Also, look over there!"
Evernote: "Here is a rational, reasoned approach behind our password requirements and limitations."
AT&T: "We don't want to give our customers the option to do something that some of them might not like."
It's almost an accurate summary. I do actually take issue with Evernote's response being rational and reasoned. They stated that they do not allow spaces in passwords because spaces in the beginning and end of a password may get trimmed and so they would need to create a validation which only allows spaces in the middle. This filter would be too much work for too little gain.

This is not a rational and reasoned approach to password management if you value security for your users. Why would you trim the password? I see no reason to remove spaces, or anything for that matter, from passwords. Just hash them as they are. Passwords should be hashed as soon as possible. The more passwords are passed around in plaintext in the code, the higher the risk. We do a hashing of the password before it is even sent to our server.

There is absolutely no reason to limit which characters are allowed in a password. Whatever stuff you type in your passwords, it should be hashed, which means that all characters become alphanumeric (base64 encoded) in most cases (with some exceptions, like bcrypt which uses $ for some field separation). So it doesn't matter if the password contains special characters or not unless your code is passing around the plaintext, unhashed password in various data structures (like JSON). This would require some parsing and transformation of the password so it doesn't mess up the structure. But you should never, ever, ever do that. Hash the password, then send the hash around instead.

Apparently these companies, Evernote included, along with a lot of other companies, are just to lazy or incompetent to handle the passwords of their own users. If you read this, make you don't do the same mistakes. Work with hashes, not plaintext passwords.

Oh, and always use good hashing + salt. SHA is bad and MD5 is even worse. Also, make sure to have one of of the salts outside the database. Just in case an attacker gains access to the database (which is far more common than access to the filesystem).

Wednesday, March 20, 2013

Beta update: playlist sync, autofill of empty titles, youtube filter and quality

Howdy!

It has been a while since my last post here. My daughter turned one year last month and even though I try to spend as much time with her as possible I have also been able to sit down a few hours with Stoffi each week for the last couple of months. The result is a new and fresh beta update with some minor, but also some larger, tweaks.

Even though I try to avoid introducing new features into beta, I just couldn't help myself. I just had to squeeze in a few cool gems. First of all you can now select the quality when watching the YouTube video. This should come in handy if you are on a connection with limited bandwidth. I have also added a filter to YouTube so it only searches inside the Music category. You can change this using a drop down in the upper right corner at the YouTube list. The last new feature is a rather simple one: if a track has an empty title I fill the title with the name of the file. This should make it easier for us with files which have bad meta data.

There's also a lot of bug fixes and minor adjustments. The keyboard shortcuts has been updated so they now include all the new navigation (YouTube, SoundCloud, Radio, etc) and I've also changed the default keys so they are easier to learn and remember.

There's also better support for regional settings of displaying time, dates and numbers in Stoffi. The Last played, Views and Installed will now display correct formatting depending on your locale settings in Windows. It even updates when you make a change in the control panel so you don't have to restart Stoffi!

When you clear a search it will go back to the previous scroll position. The cloud interface has been polished a lot (you can now manage links to Facebook, Twitter, Google, etc. directly from inside Stoffi). A bug when you pressed numpad - while renaming/creating a playlist has been fixed (thanks to andrey8688 finding it). Another bug, preventing tracks from being dropped from Explorer onto a Stoffi playlist, has been squashed (thanks to Genedon for finding it). Lastly, the seek bar is now much smoother in its movements.

So that's it. I have one more thing to fix before I am ready to send this to the general public, namely the equalizer. But while I work on that you guys should download/upgrade the beta and put it through the usual tests to make sure it's up to our standards. As soon as we can't find any more bugs and I've polished the equalizer I will mark this as stable.