Archive for the ‘ Guides ’ Category

Adding a new service/daemon manually on ubuntu

So I added a shiny new znc init script to /etc/init.d/ and I want to enable it so I can use upstart to start and stop the service and control it at different runlevels.

First, ensure the script is 755, then lets use chkconfig to get this loaded up!

If you don’t already have it:

apt-get install chkconfig

then do the following:

chkconfig –add znc
chkconfig –level 2345 znc on
service znc start

And yay!!!

I’ve solved it! How to ensure your local repo is a exact 1:1 copy of its remote!

git fetch origin
git reset --hard origin/master
git clean -dffx

If you run the above commands instead of just a git pull, your repo will be synced exactly (1:1) with its remote. This means:

1. Any new files/folders you created will be removed, EVEN SUB-REPO’s (repo’s inside your repo).
2. Any files you’ve changed that already existed in the repo will NOT have their changes stashed/saved.
3. BE CAREFUL. Something you might have been working on will get rm’ed.

Oh and as a side note, when doing git add’s, it is wise to do a “git add -A” as this will include any removals as well. So for me, how I have this setup in my zshrc:

function update () {
    cur=$(pwd); cd ~/env; git fetch origin; git reset --hard origin/master; git clean -dffx; cd $cur; cur=''; source ~/.zshrc
}

Pretty nifty huh?

Linkage that helped me: one two

Convert a Digital Ocean droplet to a VMware VM

The following guide explains a method for converting a Digital Ocean Droplet to a VMDK which can be used under VMware ESXi Hypervisor or other virtualization software.
This process is one way. It is currently impossible to convert a VMDK to a Digital Ocean Droplet.

View it here (pdf)!

Changing your irssi config when using znc

So I had the normal irssi config stuff all defined and all setup from my non-bouncer days. So I figured, with the bouncer, it shouldn’t be hard to get changed over etc..

Here is what I had pre-bouncer:

servers= {
{
    address = "server.com";
    chatnet = "derp";
    port = "1820";
    use_ssl = "yes";
    ssl_verify = "no";
    autoconnect = "Yes";
  },

);

chatnets = {
  derp = {
    type = "IRC";
    nick = "username";
    };

After much raging, I finally figured out how to configure this to use my znc account on my bouncer (which has ssl enabled and port 1234, it also supports multiple networks which are defined after your username):

servers = (
  {
    ## My bouncer!!!
    address = "znc.server.com";
    password = "username/networkname:password";
    chatnet = "derp";
    port = "1234";
    use_ssl = "yes";
    ssl_verify = "no";
    autoconnect = "Yes";
  },

);
chatnets = {
derp = {type = "IRC";};
};

EDIT: Lets say you do have another network setup with your znc bouncer. Here’s how that would look in your irssi config if you wanted to connect to both:

servers = (
  {
    ## My bouncer!!!
    address = "znc.server.com";
    password = "username/networkname:password";
    chatnet = "derp";
    port = "1234";
    use_ssl = "yes";
    ssl_verify = "no";
    autoconnect = "Yes";
  },


  {
    ## My bouncer connection for second network!!!!
    address = "znc.server.com";
    password = "username/secondnetworkname:password";
    chatnet = "anotherone";
    port = "1234";
    use_ssl = "yes";
    ssl_verify = "no";
    autoconnect = "Yes";
  },

);
chatnets = {
derp = {type = "IRC";};
anotherone = {type = "IRC";};
};

NOTE: If you have a server block but you don’t want to join that server, instead of commenting out the entire block, just change autoconnect to “no”.

and you will NOT be joined to that server. Pretty nifty if you have a ton of servers in your config but don’t want to join all of them upon irssi start.

Using Hping

hping is nice for sending a bunch of traffic just to test certain reactions of remote machines etc.. Here are some cool ways to use it and some useful links on more info.

Start a flood of icmp packets with a rand src:

hping 16.0.24.2 --rand-source --flood --icmp -V<pre>

Do 500pps with just one src:
<pre>hping 16.0.24.2 -i u2000 --icmp -V

Here, we use the -i u2000 which tells hping to send a packet every 2000us which is 500 packets per second. You can figure out what this value should be by doing “1e+6/pps”

Through the –flood option, I can get around 400kpps out of a 10G link (with those icmp packets).

Some linkage:

http://www.rationallyparanoid.com/articles/hping.html

http://www.radarhack.com/tutorial/hping2.pdf

Setup static route and arp entries on a linux box

Want all traffic destined for 16.0.24.0/24 to go through your interface with ip 192.168.192.7?

route add -net 16.0.24.0 netmask 255.255.255.0 gw 192.168.192.7

To delete this route,

route del -net 16.0.24.0/24

Ohh and static arp to one of the hosts in that network?

arp -s 16.0.24.2 00:50:49:A0:9E:E0

It’s so easy! More info here and here.

Use a proxy with Plex/plexweb using ncat

So we needed to proxy our plex server through another box. That part was easy enough with a nice little ncat socket running on the proxy:

ncat -k -l 32400 --sh-exec "ncat server.com 32400"

Now we can access through the proxy address proxy.com:32400/web which is great and all.
However, to get our devices/plexweb to use the new address, some kung fu’ery is needed.

Basically, your PMS install goes out and uses plex servers to figure out its external ip address. Hence tricking this into using another ip address could require some sort of vpn tunnel of all external traffic since PMS needs to contact through the proxy for that IP to be applied.

Or, you can do the following:

1. Install PMS on the proxy, don't worry this is temporary. 
2. Sign out of myplex on your server.
3. Start the PMS on the proxy.
4. Link your proxy with your myplex.
5. Stop PMS on the proxy. Copy your Preferences.xml from your server install to your proxy install. This file is located here: "/var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Preferences.xml"
6. Start your PMS on the proxy (now with the new Preferences.xml file)
7. Your server should show up on Plexweb. Now stop PMS on the proxy.
8. Start the ncat socket on the proxy.
9. Now, when going to the server entry on plexweb, it should show as your proxy IP and you should be able to use it just like normal. Note that your "sections" may not display correctly however you can still share them all out no problem.

* Remember to stay signed out of myplex on the actual server.

Create VirtualEnv and install Pyramid!

On ubuntu 13.10:

(you should have python3.3 installed already, it comes default)

1. wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | sudo python3.3
2. easy_install-3.3 virtualenv
3. virtualenv --no-setuptools --always-copy virtrepo/
4. wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | virtrepo/bin/python3.3
5. cd virtrepo/
6. ./easy-install pip
7. ./pip install Pyramid==1.5a4
8. virtualenv-3.3 --relocatable ../

Breakdown:
We get the latest easy-setup, install it, install the virtualenv package, create our virtualenv in a directory “virtrepo”, grab easy-setup for this new env, install pip in it, and use pip to install pyramid. We finally create relative paths in all our files so nothing is reliant in the system we built the virtualenv on.

Seriously managing ssh forwarding in virtual terminals and beyond…

Ok so this gets a little intense to think about but its something you might run into if you are using virtual terminals with ssh keys and agent forwarding etc..

NOTE: All code involved here is stored and updated at my github here!!

for all intents and purposes, tmux = screen for the rest of the article.

My first problem: When leaving a tmux session running on a server, logging out of that server and going home, logging back in and pulling up my tmux session (tmux attach), my key forwarding doesn’t work anymore. This is because your tmux session is using your old SSH_AUTH_SOCK values from when you logged in originally to create the tmux session. However, this ssh socket doesn’t exist anymore. A new one has been created when you logged in. This is simple to see from echo’ing $SSH_AUTH_SOCK in your tmux session and then again out of your tmux session and observing the difference.

To fix this, some of the common methods seem to be making a symlink in your home dir which gets updated with the newest socket to use upon login (using $HOME/.ssh/rc). Then your .bashrc or equivalent would have something like:

export SSH_AUTH_SOCK=$HOME/ssh_auth_sock

which ensures that all of your shells always point to the same symlink for the socket, which is in turn always kept up to date. More on that here.

This is super awesome. It makes your virtual terminals work forever. Except, when you are about to leave work and want to launch an automated script which will log into intranet boxes using your key, since you left work, you are no longer connected, meaning no ssh socket to use:(

The answer: ssh-agent

So we need to run a local agent (ssh-agent) to create a socket on the box that will stay no matter if we are logged in or not. This is where it gets tricky because you still have a forwarded socket created upon login which our current implementation tells our tmux session to use by default. Meaning even when you do setup ssh-agent, those changes won’t stick.

So what I needed was a bash script that can do this “registration” for me and manage it depending on the situation. The rules for this include ensuring that when I do create a local agent socket, that is used by default if available. If it is not available, we use the forwarded socket.

Here are the added lines to various files to make this all work (read the comments for info on what stuffs is doing):

To my .zshrc file (works for .bashrc as well)…note you will need to change the location of where you want the symlink to live

#Some fun ssh auth socket stuff
#setting our actual variable to pnt to symlink which is what we maintain depending on socket we are using
export SSH_AUTH_SOCK=$HOME/ssh_auth_sock

To my $HOME/.ssh/rc file (ran at ssh login, note your X forwarding won’t work using this…not sure how to fix that yet..)..You will also need to change the paths.

#!/bin/bash
if pgrep -u $LOGNAME ssh-agent > /dev/null
then
        #Well looks like ssh-agent is already running meaning the symlink is already pointing to the already created local socket...good to go!
        #Setting our backup symlink that always should point to our current remotely forwarded socket from this login
        ln -sf $SSH_AUTH_SOCK ~/ssh_auth_sock_rem
        exit
else
        #as far as we know, ssh-agent is not running.
        #lets go ahead and set this up
        if test "$SSH_AUTH_SOCK" ; then
            ln -sf $SSH_AUTH_SOCK ~/ssh_auth_sock
            ln -sf $SSH_AUTH_SOCK ~/ssh_auth_sock_rem
        fi
fi

And finally, my “registration” script which helps set up all this ssh-agent stuff…You will need to change a few variables up top depending on your paths…

#!/bin/bash
#set some vars
sshkey="$HOME/.ssh/id_rsa.work"
#the main symlink location your shell variable always points to
auth_sock_loc="$HOME/ssh_auth_sock"
#the location where your forwarded socket is
auth_sock_rem_loc="$HOME/ssh_auth_sock_rem"

 if [ "$1" == "help" -o "$1" == "-h" -o "$1" == "?" ]
 then
        echo "This will function will launch ssh-agent and ssh-add to register your private key locally on this server for continued usage when your not logged in, and when you log back in:)"
        echo "running just \"reg\" will register you making sure to first check if you are already registered...running \"reg kill\" will kill any current ssh-agent processes in event of a problem/error/corruption"
        exit
 fi
 if [ "$1" == "kill" ]
 then
        #Maybe theres a problem...this kills all ssh-agents running as youre user
        echo "Killing any running agents..."
        ps -u $LOGNAME | grep 'ssh-agent'
        killall -u $LOGNAME ssh-agent
        #setting our symlink to pnt to our remote forwarded socket
        cp -P "$auth_sock_rem_loc" "$auth_sock_loc"
        echo "Done...exiting."
        exit
 fi
 if pgrep -u $LOGNAME ssh-agent >/dev/null
 then
        echo "Yay!!! Already registered on this server ($(hostname))! Exiting..."
        exit

 else
        #Looks like we arent ssh-agented (for lack of a better word, or im just lazy) on this server
        echo "Not registered on *$(hostname)* (ssh-agent not already running)....starting up..."
        eval `ssh-agent` && ln -sf $SSH_AUTH_SOCK "$auth_sock_loc" && ssh-add "$sshkey"
        echo "Done! Your key is registered on this server. Any continuously running scripts should work successfully."
 fi

You can see from my script that it supports asking for help and another argument called “kill” that will kill any ssh-agent you currently have running (in case of corruption, etc..) then set your symlink to your currently forwarded session so you can continue on working. Running just the script itself will launch ssh-agent, setup your symlink to point to the new socket, and run ssh-add with the key you want added. Note that this entire setup relies on your key being loaded into ssh-agent (use “ssh-add -l” to see it loaded). If your key is unloaded from ssh-agent, life is gonna suck. (not really, just kill it and rerun the script).

I’m actively using this and it works beautifully. I currently have an alias to launch this script when I type “reg”. You could also just set this up to run at login so your ssh-agent is always setup whenever your on the box:)

tl;dr, copy the stuff above to files in your crap and everything is magic from there.

Installing Xen (XCP) on Debian 7

UPDATE – The Purge…

So following instructions isn’t enough apparantly. Connecting to the server with XenCenter opened my eyes.

A) To use local disks, you have to use a cli utility that doesn’t seem to work properly for me (thinks /dev/sda1 is for sure in use when it actually is not) and seems to not understand anything but an LVM setup. Now yes, I didn’t do LVM when I installed Xen but seriously?…why are there these restrictions when all I want to do is run VM’s on a decent hypervisor with a decent client/web gui?

B) When I go to create a VM in XenCenter, it shows literally nothing under the templates section which means it doesn’t let me continue with out one. Looking on t3h googles, this doesn’t seem normal at all and apparantly no one ever in the world has run into this problem. I checked and yep, the xen-guest-templates package was indeed installed.

C) Networking…meh whatever: When booting the Xen kernel, almost every other time, networking would either work, or not work. Because that is totally something I don’t need working all the time.

D)

apt-get purge xen*

E)

apt-get install kvm libvirt0

F) More to come on KVM…

——– Original Article
So until yesterday, I didn’t even know Debian 7 was out!! Since I’ve decided to use my mac exclusively as my main workstation machine, my main desktop will now become my new “quiter and power saving” server.

One of the first things I did was decide I wanted to get some virtual machines going. At first I thought ESXi….but I discovered quickly that if the installer sits at the loading kernel part for too long, it most likely doesn’t like your hardware and is trying to be nice about telling you about it.

So, I looked over to Xen. It’s free, used everywhere, and has a fairly easy time installing on ubuntu/debian based systems. Let me explain exactly how it works:

Xen is certainly a “bare-metal” Type 1 hypervisor meaning it does actually run just like ESXi does but along side of whatever base OS you installed. In our case, this will be Debian. Basically, at boot, the Xen hypervisor is booted which then in turn launches the Debian kernel etc..

Xen comes in a couple flavors. The one we will be using is the free Xen Cloud Platform (OSS and free as opposed to Xen Server, for the corporate world). We will also install the XenAPI/xapi tools enabling us to use things like XenCenter (think vsphere, a gui client for managing stuffs) or OpenXenManager.

So lets get started!

1) Install Debian 7 any way you want it…thats the way you need it. (yeah journey is awesome)

2) Install our hypervisor itself:

apt-get install xen-linux-system

3) Set Xen to be the default boot options in GRUB:

dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
update-grub

4) I personally wanted to set a memory limit for Xen or else it ballons up using as much as is available. Edit/Add /etc/default/grub with:

GRUB_CMDLINE_XEN="dom0_mem=1024M"

dropping in however much memory you’d like to allocate.
Ensure you update-grub once again.

Now you must edit /etc/xen/xend-config.sxp with:

(dom0-min-mem 1024)
(enable-dom0-ballooning no)

5) I also didn’t want Xen to utilize all my CPU’s (think core’s here as well) in the event that VM’s took up to much CPU time. To change this , add the following to the end of your /etc/default/grub file:

dom0_max_vcpus=1 dom0_vcpus_pin

and then edit the /etc/xen/xend-config.sxp file:

(dom0-cpus 1)

6) If you would like your VM’s to actually go through a shut down instead of Xen to just pause/hibernate them when you shutdown the host, edit/set these parameters in /etc/default/xendomains:

XENDOMAINS_RESTORE=false
XENDOMAINS_SAVE=""

7) I highly suggest you run a update-grub and then reboot at this point to ensure everything gets set in stone with grub and xen.

Next: Xen-tools – these are basically sets of scripts that enable you to easily create configured guest domains/hosts/vms.

1)

apt-get install xen-tools

2) edit /etc/xen-tools/xen-tools.conf for anything you might want to change with its default image creation script. In here, you can also change the directory where domU images (I presume vm’s) are stored. You can find more on the interwebz. I have not really done much yet with this but I’ll post as I learn more.

This post will be edited soon for info on xenapi.

This guide uses most of its Xen installation resources and information from here.