Category Archives: Linux

Uploading (and Resuming) videos to YouTube via GoogleCL and AWS – Updated 15th Dec

If you are like me, and have a slow and/or unreliable internet connection, trying to upload any reasonably-sized video to YouTube can be a nightmare, forcing you to have your computer on for hours on end, and then finding your upload failed because your connection dropped, and then having to start all over again.

Well, one way to have resume protection is to use a middle-point, which is Amazon Web Services, or a similar cloud-based provider, then using that to upload to YouTube. Since the connection between the cloud system and YouTube is likely to be more reliable (and faster) than your connection, the upload from the cloud system to YouTube will be faster.

The first step is to setup and start an instance on AWS. I am using the Ubuntu image.

SSH into the instance and install supporting packages via apt-get or aptitude. Make sure you change the IP (xx.xx.xx.xx) and the key (AWS_Ireland.pem) to match your files.

$ ssh -o IdentityFile=/home/user/.ssh/AWS_Ireland.pem ubuntu@xx.xx.xx.xx

$ sudo apt-get install python-gdata python-support rsync

Then download the latest googlecl deb file from https://code.google.com/p/googlecl/downloads/list

$ wget https://googlecl.googlecode.com/files/googlecl_0.9.14-2_all.deb

Now, install the deb file using dpkg

$ sudo dpkg -i googlecl_0.9.14-2_all.deb

We can now start using the Google services, but first we need to authenticate. This is normally done via a browser, but since we are in a terminal, we skip this.

$ google youtube list
Please specify user: [enter your email address here]

You will see a text-version of the login page. Don’t bother entering your values. Just press ‘q’ to quit and confirm exit. Then, you’ll see in the terminal window, a url along the lines of this:

Please log in and/or grant access via your browser at:
https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token={hidden}&hd=default

Go to that url and sign in. Then, come back to the console and press enter. If all goes well, you should see your video uploads in the console window.

Now, to upload a video to the AWS instance. You can use rsync for that, and the command to enter into your local terminal is as follows (change the key file to match yours and the IP address field to match your instance’s IP):

rsync -vhPz --compress-level=9 -e "ssh -o IdentityFile=/home/user/.ssh/AWS_Ireland.pem" source ubuntu@{EC2_IP}:.

This uploads the video called “source” onto your EC2 instance at the home folder of the default user (if you have another location in your instance, use that here). Rsync will allow you to resume uploads via the P switch. When the rsync command successfully completes, you can then SSH back onto the instance, and use the “google youtube post” command to upload your video onto YouTube.

NOTE: On some large files, rsync breaks on resuming with the error message “broken pipe”, if this happens to you, see this page (specifically, Q3).

Once your video is uploaded to your EC2 instance, you can then upload that video to YouTube by using this:

$ google youtube post path/to/video

Dust

I finished my first Steam-based game today, and it was Dust. Going to replay the final scenes again, but this time, try to catch the fight on video.

Steam

With my new laptop up and running, I am happily playing several Steam-based games:

Dust (I am pretty impressed by this. It is a platformer, side-scrolling game with really nice visuals, voice-acting and insane combo options)
Dungeon Defenders (yet to start this under Steam, but played it extensively under Android)
Dota (downloading)
Wakfu (played on my old 32-bit box, but Steam refuses to start the game if you are running 32-bit arch. With a 64-bit arch, it will start)

I also have some extra games that I’m not playing or stopped playing:
Bastion (tried it and it is interesting, but I’m finding it really difficult to get into
Dungeon Hearts (slow and laggy)
Ravensword: Shadowlands (an interesting 3D, but very little customisation options)
Toribash (Appeal wore off very quickly)

I am also playing two non-Steam-based games

World of Warcraft
League of Legends

Although, I have them tied into the Steam framework so I can start them up via Steam and grab screenshots, etc.

I have also looked at Fraps and Kazam — two screen recorders (Fraps for Windows, Kazam for Linux) and both perform pretty well. Fraps performs great, I got smooth high-quality video out of WoW. Kazam, I am tweaking settings. It also generates good-quality video, but I need to play with the framerate and encoding settings. Look out for some uploads at some point in the future.

Stealth “Turla” Malware Infects Unknown Number of Linux Systems

jamesgoode98's avatarLinux News

The Linux Turla is a new piece of malware designed to infect only Linux computers, which has managed to remain relatively hidden until now and has the potential of doing a lot of harm. Unfortunately, very little is known about it or how to fix it.

During the course of almost a year, the guys at Kaspersky Lab discovered a cyber-espionage operation which they called the “Epic Turla.” According to their research, more than 45 countries have been affected and that includes government institutions, embassies, military, education, research, pharmaceutical companies, and a lot more domains. It seems to be an organized effort, probably with a nation backing it up.

These problems were Windows-only and it looked like it will remain that way, but it turns out that “Turla” did not affect Windows systems exclusively. In fact, Linux systems are also vulnerable, to a different kind of code, which seems to…

View original post 266 more words

Docker.io Builds Page For 32-bit Architectures

I have started posting up my builds of Docker.io. They are unofficial, and unsupported by the community, pending official support and code release supporting 32-bit architectures.

https://drive.google.com/drive/u/0/#folders/0Bx5ME-up1Usbb2JMdVBvNGFSTUE

I have setup my system to auto-build every week and post to this shared directory. There’s a readme in the shared folder.

Docker.io Build Script Update

Some changes to the Docker.io code has caused the build script to fail, this was down to the code now using btrfs to build a driver. It has taken me a while to figure out how to fix that error message, but the script now works. You have to add this chunk of code anywhere before the main docker build

git clone git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
mv btrfs-progs btrfs #Needed to include into Docker code
export PATH=$PATH:$(pwd)
cd btrfs
make || (echo "btrfs compile failed" && exit 1)
export C_INCLUDE_PATH=$C_INCLUDE_PATH:$(pwd) #Might not be needed
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:$(pwd) #Might not be needed
echo PATH: $PATH
cd ..

How Splitting A Computer Into Multiple Realities Can Protect You From Hackers

Virtualisation, Sandboxes, Containers. All terms and technologies used for various reasons. Security is not always the main reason, but considering the details in this article, it is a valid point. It is simple enough to setup a container in your machine. LXC/Linux Containers for example, don’t have as much overhead as a VirtualBox or VMWare virtual machine and can run almost, if not just as fast as a native installation (I’m using LXC for my Docker.io build script), but conceptually, if you use a container, and it is infected with malware, you can drop and rebuild the container, or roll back to a snapshot much more easily than reimaging your machine.

Right now I run three different containers — one is my main Ubuntu Studio, which is not a container, but my core OS. the second is my Docker.io build LXC, which I rebuild everytime I compile (and I now have that tied into Jenkins, so I might put up regular builds somehow), and the final one is a VirtualBox virtual machine that runs Windows 7 so I don’t have to dual boot.

How Splitting A Computer Into Multiple Realities Can Protect You From Hackers | WIRED.

Kernel Tinkering, SuSE, and SteamOS

This is what I like about studying for certifications. They force you to look into subjects at a deeper level than you may otherwise have done. One of the topics in LPIC-2 is Kernel maintenance – understanding the kernel, how it works, the concept of dynamically loaded modules, compiling the kernel and modifying the configuration prior to compiling. It is very intriguing learning about this low-level part of the Linux OS.

It is a complex topic though, and compiling a kernel can take a while, depending on configuration. I managed to compile and install my first kernel today. :)

Meanwhile, since I’ve now obtained my SUSE 11 CLA from Novell (not sure if anything will come through in the post, though), I must really refresh my memory on openSuSE. My primary distributions have been Ubuntu and Fedora.

Whilst downloading various linux distributions including the latest versions of Ubuntu and Fedora, I found out that I could download SteamOS, which is Valve’s own distribution for running its Steam platform. Problem is it only supports x64 which I don’t have (yet)

Microsoft Appeals to Developers, Developers, Developers | Linux.com

It would appear Microsoft are looking to port .NET onto Linux and Mac. Whilst Linux already has a replacement open source framework for that, this news is obviously raising eyebrows and questioning Microsoft’s long term goals.

Former Microsoft CEO Steve Ballmer became infamous in 2006 after leading a Microsoft Windows meeting in a chant, “developers, developers, developers.” While the images of him clapping his hands and screaming became the target of the early social media and YouTube culture, he was right with his intention. Developers are the masters of the universe (at least in the world of software), and Microsoft gets it.

Today the company is making a rather big announcement: It is open sourcing the server side .NET stack and expanding it to run on Linux and Mac OS platforms. All developers will now be able to build .NET cloud applications on Linux and Mac. These are huge moves for the company and follow its recent acknowledgement that at least 20 percent of Azure VMs are running Linux. This struck a chord in the Twittersphere but wasn’t all that surprising when you consider how pervasive Linux is in the cloud.

Microsoft Appeals to Developers, Developers, Developers | Linux.com.

 

LPIC-1

Linux Professional Institute Logo

Exquisite-suse

Nearly two weeks after I obtained the qualification, I finally got the physical version in the post – and now also doubled up as a SUSE CLA.