Sunday, 21 June 2015

About me:
Short resume (isn't that redundant):

Age: Thirtish (but not for long).

Education: 
      • Major in physics.
      • Advance studies course: "Technological Applications of Lasers".
      • 3-year residency program in Medical Physics.
      • 9 year experience as a radiation protection officer and medical physics expert.
IT:
      • Computer guru (wannabe).
        • Linux (debian).
        • Windows (almost every flavour). 
      • Networking (basic skills).
      • Programming: python, vb, java, c#. (Software developed by me is running on production enviroment at my jobplace).
Languages:
      • Galician (mother tongue).
      • Spanish (mother tongue).
      • English (C2 acording to Cambridge evaluation).
      • Portuguese (Read and understand).
      • French (Basic skills).
      • Italian (not really) .
      • Mandarin (have just started).

Research:

      • 2 years biomaterials research at university.
      • Have published several jobs on national research magazines and congresses (international research coming soon, or not...)
Interests:
      • Physics (either medical or not).
      • Maths.
      • Computer science. 
      • Gadgets.
      • Books.
      • Movies.


Yowdaemon whatsapp remote control for the servers:



I bring you here a small (yet useful piece of software). You probably have read something about having whatsapp on your computer using the Yowsup libraries. Buliding upon that I’ve created a small program to work as a remote communication method with your computer or server. I’m using it with my raspberry (because the daemon is super-lightweight) but it will run on any *nix system. (Please tell me if you run it on a mac os).
Whatsapp is a messenger service, which links to a phone number, and sends unlimited messages, pictures, etc… (for a yearly fee). If you didn’t know this you probably don’t know what internet is either (no, I don’t mean the net inside your swimming pants). It is meant to be used on your phone overriding it’s sms’ capabilities. There are official “whatsapp clients” for android, ios, windows phone, Blackberry, Symbian (is someone still using this!!!) and Nokia S40 (WOW!!). There are a few unofficial clients out there (like wazapp for the nokia N9, also based on yowsup).
Afaik the yearly payment is based on the usage of the official client, so no fee for the unofficial ones :).
Yowsup are a set of libraries developed by Tarek Galal in Python. Yowsup allows the use of the whatsapp messenger service on any computer with python capabilities (mostly anything). Right now yowsup is under heavy development.
Python is Monty Python-inspired (//en.wikipedia.org/wiki/Python_programming_language) programming language. It “is a widely used general-purpose, high-level programming language” (according to Wikipedia(www.wikipedia.com).
Seriously? You seriously expect me to write this…
Yowsup-daemon:
Yowsup is centered on using whatsapp on your computer like you would in your phone
Yowsup-daemon is a small program which (using yowsup) allows you to communicate with your computer. There are a few uses that I can think of:


1    Tell your computer to do something: for example download a file, start a backup, check if some service is still running… This could include checking your home temperature (given that you have a temperature probe attached (extremely unexpensive)) and turn on your heating (maybe some arduino is needed here).


1      For your computer to tell you something: like some task was finished, someone has connected to your computer, some download just finished… 
      Features: 
    
  • Since it is server-based there is no need to open ports on your router. 
  • The security is blindlessly handed to the sever side (is this good? I don’t think anyone is going to hack your servers through whatsapp in the near future). 
  • It doesn’t establish a link. It just sends messages. This is really good for me because on my long (underground train) commute my connection drops every 5 mins. And my tasks don’t get stopped (and I save myself the nohup hassle). 
  • It works as a daemon so if you make it run at startup you can forget about it.
  • Impress your friends or clients. Your (technologically impaired) client might not want to connect through ssh to check the status of his server but he will be pleased if the same can be achieved through whatsapp. Since it is fully open source you can check the code and be sure it will not harm your system.
Could the same thing be achieved with email? Yes, but the security would be worse and it would be way uncooler.
Getting started:

1.       First you have to download yowsup from Tagal's github.
2.       You need to register your phone number into whatsapp…
3.    Get it running, check Tagal's instructions. Ask me for help if you run into trouble here.
4.    Download things from my github.
3.       Create your config file; start with the template: 
  • Every line starting with # (or empty) is ignored. 
  • Allow_direct_shell: if this keyword is present shell orders will be accepted; for example: “Shell: whoami”. Notice: the colon and the space. 
  • Allowed phone numbers: put as many as you want (I only tried 3 or 4). Remember complete phone number with country code. 
  • #Command dictionary: after this line the commands are defined. First keyword, then colon and then the command to execute. Anything that is command-line-acceptable is acceptable here. If you want to call a script remember to call it with full path. I give you a few examples working for me.
  • Start it with: ./yowdaemon.py start
An example config file is:  
#This is the config file
#Read the instructions, read my blog www.pisasuavemente.blogspot.com/, tweet me @treadsoftblog

#Your computer's (Whatsapp) phone number.
phone number:"Your phone number, with cc"

#Your whatsapp password.
password:"Your password goes here"

#Uncomment the following line to allow direct shell commands.
allow_direct_shell

#Uncomment the following line to send a message to the allowed1 number.
send_message_startup:Yow-daemon started.

#Uncomment the following line to reply to groups.
answer_to_group

#Message to answer when the order was not understood.
not_understood: No te he entendido.

#Uncomment the next line to activate guest mode.
guest_mode

#allowed phone numbers
allowed1:"Dont forget these"
allowed2:
allowed3:

#uncomment the following line to log.
logging

#Command dictionary
Bluetoothon:service bluetooth start
Bluetoothoff:service bluetooth stop
Bluetooth:service bluetooth status
Music play:mpc play
Music: mpc
Music_stop: mpc stop
Say: fortune
Di algo: fortune
Hi: Hi
Correo:python /home/pi/Documents/a.py
Reload_dlna: service minidlna force-reload
Free:
Cputemp:/opt/vc/bin/vcgencmd measure_temp
Cput: /home/pi/meteo/check_Temp
Temp: /home/pi/meteo/temp
Pres: /home/pi/meteo/bmp.py

Sunday, 16 November 2014

Housekeeping: Debian backup (raspberry).

Recently i was messing around with my Raspberry Pi and i totally screwed my system.
After restoring my system to a functional level i decided to do some housekeeping chores.
By housekeeping i mean doing some backups and getting prepared for restoring them.

I'm writing this post mainly as a future reference. All of the contents are explained elsewhere (i mean i dont claim their authorship) and are public knowledge.

First get yourself plenty of storage, i would advise to get an external (independently powered) hard disk, but if you have a big enough sd card the process will be pretty similar.

I really like using the dd command (on wikipedia they say it stands for disk destroyer or delete data, that would really make sense). Its a powerfull low level command. On the first place it is not a good idea to copy a live system with dd so you should start by starting from another partition to do the backup. That is why i decided i needed a multiboot system. Noobs and berryboot really mess the filesystem so i decided to have multiple system partitions and boot from them chosing with the cmdline.txt file.


First we will need to have two (or maybe three) equally sized partitions (at the end of the post i'll explain how to overcome this). I really advise to format the partitions on ext4 filesystem. I haven't tried using btrfs (or any other fancy fs out there) but any intensive use on ntfs is really slow for the little Pi. I will call your partitions sda1, sda2 and sda3 (if you have a multipartition sd card you will have mmcblk0p2, mmcblk0p3, mmcblk0p4). Remember your boot partition will always be mmcblk0p1 We will assume sda1 and sda2 are equally sized partitions (roughly 5 gigs) and sda3 is a much bigger storage partition.

On a linux PC (use a live one if you dont have a linux installed on your PC) first copy your system partition sda1 over sda2. For this i use gparted graphical interface but you could use dd as we explain latter.

Now you have two system partitions and you could start any of them just by editing the cmdline.txt file on /boot. 

BEWARE of a few catches: 
  • before you can start from sda2 you first need to mount it and edit fstab file so that the root partition is sda2 and not sda1 (no need to touch anything else there).
  • When the system boots (before it connects to the internet, if that is even possible) the system time will get loaded from a file callled /etc/fake-hwclock.data. Your system might detect that your partitions have been mounted sometime in the future. The Pi will think it is due to filesystem corruption and will stall until you confirm that you want to do a diskcheck. If you dont have a keyboard (called running headless) you are in trouble. The solution is to manually change fake-hwclock.data to sometime in the future. If the problem has already appeared you will have to unplug your hdd (my pi and its hdd are 10 mins deep under the TV) and do it in your ext4 capable pc (i mean linux) or plug an external keyboard (my boss looks at me funny when he sees me stealing keyboards from work).
So at this point you have two equally capable partitions. For the price of hdds these days i would advise having three system partitions. In my case i have: 
  1. Stable system (sda1).
  2. Service (for backups and stuff) (sda2)
  3. Test (for installing potentially harmfull or system-breaking stuff) (sda3)
Ok. now you have to boot from your service partition and start with the backup fun. So you have a sda2 mounted as a root partition. Your storage partition (sda3) is mounted in /media/whatever (could even mount a network disk but performance most probably be sluggish).
To copy your partition you use:

sudo dd if=/dev/sda1 of=media/whatever/image.img conv=sync, noerror

This will copy (bit by bit) whatever is in your sda1 partition, conv=sync, noerror means that any error reading a bit will be written with the null value so disk structure is kept.

 If it is the first time you do this you might want to stop it after a minute or so with Ctrl+C to see what is the speed so you can calculate how long is it going to take.

It took a little bit north of 15 min to copy my 5GB partition. Since it is a bit by bit copy you will get a 5GB file (the size of your partition) so you might want to pass it through gzip:

sudo dd if=/dev/sda1 conv=nosync,noerror | gzip -c > /media/whatever/image.img.gz

This took 90 mins (about the length of a movie) and gave me a 2GB file.

But here I remembered something. When you delete a file the information is still there, files are just removed from file indexes. So the bits are still there when you dd the disk and feed it to gzip you are feeding all those deleted files. What can be done? Well we can restore the system to a maximum order (reduce entropy, nice word: entropy. Use it whenever you can to make you look smarter). So we create a file composed of all zeros to fill up your disk:

first mount the partition that you want to fill with zeros. The mount point has to be created first (if it wasn't: sudo mkdir /media/test):

sudo mount /dev/sda1 /media/test


dd if =/dev/zero of=delete.me bs=8M

This creates a  file made up of zeros. When it stops because disk is full you delete the recently created file:

sudo rm delete.me

unmount the partition and repeat the dd (wash, rinse and repeat...). Hey create a file and then delete it just to reduce entropy (for those worried for the laws of thermodynamics know that to reduce your hdds entropy universes entropy has been increased.

sudo umount /dev/sda1

sudo dd if=/dev/sda1 conv=nosync,noerror | gzip -c > /media/whatever/image.img.gz

This time it took only an hour and it generated a 1.1 GB file.

Now that we are at it:

If you have some nasty secrets that you deleted you know anyone can undelete them. If you cover your tracks with zeros not everyone can undelete them, but CIA and NSA can. so if you cover your files with zeros (delete) and repeat with ones (up to 8 times) you can be fairly sure no one will be able to reccover your dirty secrets.

You think this is useless: Well... some politicians in my country don't know this and are in jail because of it.

OK we have a backup on a .img.gz file what can we do with it? 

Imagine you have messed up, deleted or otherwise rendered unusable your system.  you take your sd card to your PC, change your root partition to sda2 and boot normally. Then you have to restore from your .img.gz file:

gzip -dc /media/whatever/image.img.gz | dd of=/dev/sda1

If it wasn't zipped: 

dd if =/media/whatever/image.img of=/dev/sda1

I would recommend that you try that your backup is working on one of your unused partitions (sda2, sda3) before you really need it.

Hey, I want a file from my backup but i don't want to go through all the hussle of copying the whole partition and boot off of it:

No prob there:

mount -t ext4 -o loop /media/whatever/image.img /media/test

But you said that I should compress my file.

Ok, afaik you shoul decompress you file and then mount it.

If you messed up your sda3 (test) partition and want to restore it you might want to copy sda1 over sda3:

sudo dd if=/dev/sda1 of=/dev/sda3 conv=sync, noerror 

Before we leave it:

What happens if you have a 5 gig partition and you want to copy it over your 4 gig card (of course all the data must fit, i mean more than 1 gig free space on the 5gig one). Dont worry, it can be done.

You have to resize your partition:

First check the filesystem for errors:

e2fsck -f

Then resize the filesystem. Remember the partition holds your partition:

resize2fs /dev/sda3 3.5GB

Willl yield a 3.5 GB filesystem. You can make a direct copy to the smaller partition or to a file.

After the copy you could resize your filesystem:


resize2fs /dev/sda3 

This will swell your filesystem to the size of its partition. 

If you wanted to shrink the partition you would have to shrink the fs (like we just saw) and then use fdisk:

sudo fdisk -l

lists the partitions

sudo fdisk /dev/sda 

will get you in fdisk. You have several options:
  • m help.
  • d delete partition
  • n create new partition
To shrink (or enlarge a partition you'll have to delete it and recreate it with the aproppiate size.
As a last bonus, to create a filesystem on a newly created partition use:
 
mkfs.ext4 -L Disklabel /dev/sda3

So what if you want to try an image meant to be flashed on your SD card:

First you'll have to check it with fdisk:

fdisk -l /media/whatever/image.img

Saturday, 29 March 2014

Bluetooth range booster (and wifi).



Recently i got me a pair of bluetooth headphones, my purpose was to use them with my cell phone. It came as a surprise that the sound was choppy (not sure if it's the right word but the sound stutters). with the phone on my pocket and the earphones on my head (yes i like to do it that way, i tried with the phone on my nose and the earphones on my feet and, somehow,  it didn't fell right). I checked the earphones with my ipad and the range was correct, around 10m (that is 33 feet).
I suppose my cell phone is the one to be blamed, it is a chinese made Zopo ZP980 (codenamed Potato by my wife). So what are my choices?
On the first place i decided to open up my phone and take a look.
This is where the antenna should be.
##################WARNING-DISCLAIMER: ###########################

What you are about to read will most likely render your phone useless, it's probably ilegal and bamby's mother's killers1 are supposed to have performed this procedure. Proceed at your own risk, i decline any responsability for the damages.

########################################################################

After opening the phone I saw where the antenna for the bluetooth (and wifi) was supposed to be. there was a sticker (see picture) where the antenna was suppossed to be. The antenna was absolutely useless (if you ask me that is not an antenna, only the connection). Then i remembered that the wifi signal was between poor and worse.
if your phone doesn't have an antenna what should you do, you should build yourself one.
what is the wavelength of bluetooth?? Well, bluetooth's frequency is 2,3 GHz. That is aprox. 13 cm, so a quarter wavelength would be 3 cm (1,2 inches). I took a cable of 2,5 cm (1 inch) length (i suppossed that the track and connector will make up for the 5 mm (0,2 inches) missing. The cable was sticked on the connector (see pic), no soldering or glue was used.

That red wire is acting as a supplementary antenna!!!

The result on my case YMMV2 was that both bluetooth's and wifi's signal was boosted tenthfold.
Bluetooth is useful at 9 m distance (10 yards) and wifi's usable on my whole home (before the fix the signal died everywhere except the living, where the router is).
My results are absolutely not scientific, i only tried on my cell. I should check bluettoth's performance varying the length of the cable used as antenna and determine the optimum's length. I will not do that because something that I have previously probed is that tinkering too much with your electronics equipment wil end up breaking it. I try to reduce my tinkering to a minimum.
The antenna length would only be useful for this model because the cooper length on the circuit's board should be taken into account.
the moral of the story is that if your bluetooth's signal is very bad it could be blamed on the antenna: defective or nonexistent. In that case it might be fixed (might, might might) at home through a diy procedure. The one described above is extremely simple but needs some basic skills that can break your phone.
Remember that you probably can live without a bluetooth (or wifi) connection but you probably won't be able to go through without a working phone.
Good luck and please comment if you think the info was useful to you, specially if you can reproduce the results (on this same model or other).
1 Sorry, in Spain Bambi's mother''s killers represent everything that is evil.
2YMMV - Your mileage may vary