Comparing sed stream output in linux

Sed is very very powerful, which is a good thing to be aware of.
I was looking to compare the output of a sed command to the original file before I wanted to execute the sed command directly on the file and came across this handy trick.
It works by using temporary named pipes inside the diff command.
Contents of file:
[code title=”contents of numbers.txt:”]
One
Two
Three
Four
Five
[/code]
If I just want to remove the line which begins with “Four”, I can check my sed command like this:
[code title=”Terminal output:”]
joris@beanie ~
$ diff <(sed ‘/Four/d’ numbers.txt) numbers.txt
3a4
> Four
[/code]
Awesome possum, now I know my sed command won’t destroy anything.

SSH Remote Execute command, multiple command and with interaction

This article shows how to execute remote commands via ssh, but you’ll send the commands from your own shell.
[code highlight=”1″]
ssh my_server ‘ls -l /home/my_home_dir’
[/code]
This will result in this output:
[code highlight=”1″]
$ ssh ae2 ‘ls -lha ~’
total 36K
drwxr-xr-x 2 joris joris 4.0K Jan 23 11:42 .
drwxr-xr-x. 5 root root 4.0K Jan 23 11:41 ..
-rw-r–r– 1 joris joris 54 Jan 23 11:41 .bash_logout
-rw-r–r– 1 joris joris 507 Jan 23 11:41 .bash_profile
-rw-r–r– 1 joris joris 213 Jan 23 11:41 .bashrc
-rw——- 1 joris joris 51 Jan 23 11:42 .history
-rw-r–r– 1 joris joris 171 Jan 23 11:41 .kshrc
-rw-r–r– 1 joris joris 375 Jan 23 11:41 .profile
-rw-r–r– 1 joris joris 153 Jan 23 11:41 .vimrc
[/code]
What’s even better, is that you can run multiple commands separated with a semi colon, like this:
[code highlight=”1″]
ssh my_server ‘ls -l /home/my_home_dir;whoami’
[/code]
And the best trick is this one, user input with an interactive command, sending input and output back and forth!
[code highlight=”1″]
ssh -t my_server ‘vi ~/.bash_profile’
[/code]

Oracle SQL Developer 4 does not run on Oracle Java 7 on Ubuntu 14.04

Wow, ain’t this awkward :-). I cannot run Oracle SQL Developer 4 (4.0.2) on Ubuntu with Oracle JDK 7..

To be complete: when running SQL Developer with JDK 7 from Oracle itself, displays the following error;

joris@dipshit:~/programs/sqldeveloper$ ./sqldeveloper.sh
Oracle SQL Developer
Copyright (c) 1997, 2014, Oracle and/or its affiliates. All rights reserved.
LOAD TIME : 968#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x6aa69be0, pid=9537, tid=1836366656
#
# JRE version: Java(TM) SE Runtime Environment (7.0_65-b17) (build 1.7.0_65-b17)
# Java VM: Java HotSpot(TM) Server VM (24.65-b04 mixed mode linux-x86 )
# Problematic frame:
# C 0x6aa69be0
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/joris/programs/sqldeveloper/sqldeveloper/bin/hs_err_pid9537.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
#
/home/joris/programs/sqldeveloper/sqldeveloper/bin/../../ide/bin/launcher.sh: line 1193: 9537 Aborted (core dumped) ${JAVA} "${APP_VM_OPTS[@]}" ${APP_ENV_VARS} -classpath ${APP_CLASSPATH} ${APP_MAIN_CLASS} "${APP_APP_OPTS[@]}"

Solution: Run Oracle SQL Developer with OpenJDK

First we’ll need to install OpenJDK:

sudo apt-get install openjdk-7-jdk 

Then we’ll need to change the path which SQL Developer uses. This was asked once when you first started it and it is saved in the following path:

~/.sqldeveloper/4.0.0/product.conf

The file [[ product.conf ]] contains the value SetJavaHome, we need to change this to the OpenJDK path;

If you're running 32 bit Ubuntu:

SetJavaHome /usr/lib/jvm/java-7-openjdk-i386

Or if you're running 64 bit Ubuntu:

SetJavaHome /usr/lib/jvm/java-7-openjdk-amd64

After saving this change, you can start SQL Developer on Ubuntu 14.04 and it will use OpenJDK 7, without changing your regular Java settings!

SSH without password

SSH is one of the most friendly deamons in the Linux toolbox, you can port forward your home server, you can surf the internet via your own proxy server, you can transfer files, it’s the first thing I set up on a freshly installed box and by far the most used service around my home (yeah, that’s nerdy) 🙂
This post will outline how you can create a public and private RSA key pair, and then we will use that key pair to authenticate ourselves to another computer in the network.
We will need to complete the following steps:

  1. Create a public/private rsa key pair
  2. Copy the public key to the remote host via ssh-copy-id
  3. Login to the remote system without a password

Create a public/private rsa key pair

If you haven’t created a rsa key pair yet, we can create it with this command:

ssh-keygen

As shown in this screenshot:
Screenshot from 2014-07-28 21:28:18If you have previously created a key pair, you will be asked to replace this.
For my home machines I don’t use a passphrase, if you’re more paranoid (or careful) you can do so. Please read this article for more info on passphrases and how to use them.
The keys have been generated in /home/yourloginname/.ssh and are called id_rsa and id_rsa.pub.
Never, never send someone your private (id_rsa) key! That is the same as handing over your house keys..

Copy the public key to the remote host via ssh-copy-id

This step should be repeated for all hosts to which you want to SSH with the newly created RSA key pair.
The command used is:

ssh-copy-id -i ~/.ssh/id_rsa joris@192.168.0.10

Screenshot from 2014-07-28 21:30:35
 
You will need to enter your password one last time, after that your public key is added to the authorized_keys file, which is automatically created on the target system.
After this step you don’t need to use a password any more!

Login to the remote system without a password

Just SSH into the remote system:
Screenshot from 2014-07-28 21:32:01
And wonder what you’re gonna do with all that extra time you’ve just won because entering passwords belongs to the past… 🙂
 
 

CHere Bash Here without Admin Rights – CYGWIN

I wanted to add a “Bash Here” context entry when I right click on a directory:
Screenshot - 28-2-2014 , 13_01_20
This Bash Here would open that directory in Cygwin (mintty).
My current PC is fairly regulated, so without admin rights and thus I can’t use the CHere option which I would normally use.
I just created the registry keys via regedit and that works fine.
You can use two options:

  1. Import the registry file mentioned below
  2. Create the keys yourself

1. Import the registry file

Create a file called cygwin_bash_here.reg with the following contents:

Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Classes\Directory]
[HKEY_CURRENT_USER\Software\Classes\Directory\shell]
[HKEY_CURRENT_USER\Software\Classes\Directory\shell\bashhere]
@="Bash Here"
[HKEY_CURRENT_USER\Software\Classes\Directory\shell\bashhere\command]
@="C:\\cygwin\\bin\\mintty.exe"

Save the file and double click on it to import it to the registry.

2. Create the keys yourself

  1. Open the registry editor: Start – Run – “regedit”
  2. Open the following path: HKEY_CURRENT_USER\Software\Classes
  3. Create key: “Directory”
  4. Inside Directory, create key: “shell”
  5. Inside shell, create key: “bashhere”
  6. Inside bashhere double click on the standard key and insert text “Bash here” without quotes (This is the text which is displayed in the context menu.
  7. Inside bashhere, create key: “command”
  8. Inside command, double click on the standard key and insert text “C:\cygwin\bin\mintty.exe” without quotes. Important: This is the command which is run. Please change directory to your cygwin directory

Screenshot - 28-2-2014 , 13_16_02

Conclusion

Both options will create a context menu item which opens Cygwin at your current windows directory!

Cygwin – Your group is currently mkpasswd

This message occurs every time you start your freshly installed Cygwin when you’re logged in as a domain user.

Your group is currently "mkpasswd".  This indicates that your
gid is not in /etc/group and your uid is not in /etc/passwd.
The /etc/passwd (and possibly /etc/group) files should be rebuilt.
See the man pages for mkpasswd and mkgroup then, for example, run
mkpasswd -l [-d] >> /etc/passwd
mkgroup  -l [-d] >> /etc/group
Note that the -d switch is necessary for domain users.

Important: You need to install Cygwin with the user you are logged in with.
Tip
: Remove the word “setup” from the cygwin executable to be able to install it without administrator privileges. (e.g. setup-x64.exe should be renamed to cygwin-x64.exe)

  • mkpasswd -l only shows my local users, and not the domain user I’m logged in with, so that does not solve this.
  • mkpasswd -l -d get an enormous amount of users because it tries to replicate my whole organisation, which is not necessary.

We just need our current user ( mkpasswd -c ) to be sent to the /etc/passwd and /etc/group files, to do this, we use this command:

Solution:

mkpasswd -c >> /etc/passwd
mkgroup -c >> /etc/group

After that, our current account is added to both /etc/passwd and /etc/group and the annoying greeting message is gone!

SSH through a proxy to a remote server

I wanted to SSH into my home server from my workplace but I couldn’t reach it directly because of the way the network was set up.
As it turns out it is quite easy to do by using the corkscrew program.
Edit ~/.ssh/config and add the following lines:

Host home joris.his.homeserver.com
    Hostname joris.his.homeserver.com
    User joris
    ProxyCommand corkscrew proxyserveraddress proxyserverport %h %p

The most important part is the ProxyCommand, this lets your ssh client know that it should use corkscrew as a proxy to your host. %h means the host of your remote server, %p means the port of your remote server.

Run UltraVNC as user, prevent "No Password"

I wanted to run UltraVNC on my laptop to share the screen towards my desktop computer. I have no administrator rights on this pc but I can run the standalone UltraVNC server fine.
To run it without admin rights, download the zip file from the latest version from here, choose the “bin zip” downloads for your OS at the bottom of the page.
When you run winvnc.exe, the following message presented itself in a pop-up:

No password has been set & this machine has been preconfigured to prevent users from setting their own.
You must contact a System Administrator to configure WinVNC properly.

WinVNC No Password popup
Screenshot – WinVNC No Password popup

This means you need to set the password first, but you need to run WinVNC to be able to set the password (kind of a Catch 22 there)
Add the contents below in a new file called ultravnc.ini in the same folder as winvnc.exe and your password is set to “nopassword” (without the quotes). You can now startup WinVNC by doubleclicking winvnc.exe.

[Permissions]
[admin]
FileTransferEnabled=1
FTUserImpersonation=1
BlankMonitorEnabled=1
BlankInputsOnly=0
CaptureAlphaBlending=0
BlackAlphaBlending=0
DefaultScale=1
UseDSMPlugin=0
DSMPlugin=
DSMPluginConfig=
primary=1
secondary=0
SocketConnect=1
HTTPConnect=1
XDMCPConnect=0
AutoPortSelect=0
InputsEnabled=1
LocalInputsDisabled=0
IdleTimeout=0
EnableJapInput=0
QuerySetting=2
QueryTimeout=10
QueryAccept=0
LockSetting=0
RemoveWallpaper=1
RemoveEffects=0
RemoveFontSmoothing=0
RemoveAero=1
DebugMode=0
Avilog=0
path=C:\development\programs\UltraVNC
DebugLevel=0
AllowLoopback=0
LoopbackOnly=0
AllowShutdown=1
AllowProperties=1
AllowEditClients=1
FileTransferTimeout=30
KeepAliveInterval=5
SocketKeepAliveTimeout=10000
DisableTrayIcon=0
MSLogonRequired=0
NewMSLogon=0
ConnectPriority=0
PortNumber=5900
HTTPPortNumber=5800
[ultravnc]
; both passwords are "nopassword"
passwd=33117E54AA0D4D3B55
passwd2=33117E54AA0D4D3B55

Important: Do change your password in the settings when you can run WinVNC!

Screenshot - WinVNC Administrator Settings
Screenshot – WinVNC Administrator Settings

Format any XML (windows)

With this blog post, it’s possible to format any xml you’ve selected.
It will take about 5 minutes to get this working, a couple of manual steps are necessary.
These are the tools being used:

Steps to get it working: (All steps are necessary!)

  1. First install AutoHotKey_L and start it, you’ll see the logo if it’s running: AutoHotKey logo
  2. Create a folder on your Hard Drive called “C:\autohotkey”
  3. Create a file “tidycfg.ini” in the location “C:\autohotkey\tidycfg.ini” with the following contents:
    1. [Clean Indent XML]
      input-xml: yes
      bare:no
      clean:no
      fix-bad-comments:no
      fix-backslash:no
      indent:yes
      indent-attributes:no
      indent-spaces:4
      char-encoding:raw
      replace-color:no
      wrap:0
      wrap-asp:no
      wrap-jste:no
      wrap-php:no
      write-back:yes
  4. Place tidy.exe in the location C:\autohotkey
  5. Create a file called “autohotkey.ahk” in C:\autohotkey with the following contents:
    1. #x::
      sleep 50
      Send ^c
      ClipWait
      clipboard = %clipboard%
      FileDelete, C:\autohotkey\format_with_tidy.xml
      FileAppend, %clipboard%, C:\autohotkey\format_with_tidy.xml
      sleep 50
      RunWait, %comspec% /c C:\autohotkey\tidy.exe -config C:\autohotkey\tidycfg.ini C:\autohotkey\format_with_tidy.xml
      clipboard =
      FileRead, clipboard, C:\autohotkey\format_with_tidy.xml
      Return
  6. Doubleclick the file “autohotkey.ahk” in C:\autohotkey
  7. Select unformatted xml and press [ WINKEY+X ]
  8. The formatted xml is now in your clipboard and can be pasted anywhere.

Extra information:

These are the steps what is happening in the autohotkey file:

  1. Configure the keyboard shortcut (# = winkey, x = x-key)
  2. Wait 50 milliseconds
  3. Send the ctrl-c with keyboard combination ^c ( ^ = ctrl, c = c-key)
  4. Wait until the clipboard is filled
  5. remove all formatting from the clipboard
  6. Delete the temporary file (if present)
  7. Paste the clipboard contents in the temporary file
  8. Wait 50 milliseconds
  9. Run tidy.exe on the temporary file with the configuration settings
  10. empty the clipboard
  11. Paste the contents of the temporary file in the clipboard
  12. End script