Log as well as see error and standard output in Linux Terminal

Quick snippet, as I was searching for this one today.

You can output both standard and error output to both screen and log file with the below command:

my_command.sh 2>&1 | tee my_command.log

Explanation:

  • my_command.sh
    • replace with your command (possibly including arguments)
  • 2>&1
    • redirects standard error to standard output, otherwise only standard output is written to file
  • | tee
    • my_command.log pipes the output to tee, which writes both to screen and file

Remove duplicate lines while comparing two files

I’ve been quite busy this whole day with a partially complete database dump and wanted to prepare for tomorrow with some ninja bash voodoo shizzle. I’m doing a braindump here because I know I’ll have forgotten this when I wake up tomorrow 🙂
The command stated below was the first working example I’ve gotten together, please let me know if you know a neater / better solution!

The situation:

I’ve got two files. The first file contains lines which need to be deleted from the second line (if they exist there) Continue reading “Remove duplicate lines while comparing two files”

Comparing sed stream output in linux

Sed is very very powerful, which is a good thing to be aware of.
I was looking to compare the output of a sed command to the original file before I wanted to execute the sed command directly on the file and came across this handy trick.
It works by using temporary named pipes inside the diff command.
Contents of file:
[code title=”contents of numbers.txt:”]
One
Two
Three
Four
Five
[/code]
If I just want to remove the line which begins with “Four”, I can check my sed command like this:
[code title=”Terminal output:”]
joris@beanie ~
$ diff <(sed ‘/Four/d’ numbers.txt) numbers.txt
3a4
> Four
[/code]
Awesome possum, now I know my sed command won’t destroy anything.

Bash: Shortcuts to your favorite directories with CDPATH

This is a fairly handy trick.
I’m always logging in to different servers, which all have a different location for their logs. With this trick you can login and just type “cd logs” from anywhere.
First we will add our directory to the CDPATH variable:
[code language=”Bash”]
joris@badattitude /data/share/my_domain
$ echo $CDPATH
joris@badattitude /data/share/my_domain
$ export CDPATH=/data/share/my_domain
joris@badattitude /data/share/my_domain
$ echo $CDPATH
/data/share/my_domain
[/code]
Then we’ll check if it is working:
[code language=”Bash”]
joris@badattitude ~
$ cd logs
/data/share/my_domain/logs
joris@badattitude /data/share/my_domain/logs
$
[/code]
Awesome, we’ve arrived in our logs directory straight from our homedir!

SSH Remote Execute command, multiple command and with interaction

This article shows how to execute remote commands via ssh, but you’ll send the commands from your own shell.
[code highlight=”1″]
ssh my_server ‘ls -l /home/my_home_dir’
[/code]
This will result in this output:
[code highlight=”1″]
$ ssh ae2 ‘ls -lha ~’
total 36K
drwxr-xr-x 2 joris joris 4.0K Jan 23 11:42 .
drwxr-xr-x. 5 root root 4.0K Jan 23 11:41 ..
-rw-r–r– 1 joris joris 54 Jan 23 11:41 .bash_logout
-rw-r–r– 1 joris joris 507 Jan 23 11:41 .bash_profile
-rw-r–r– 1 joris joris 213 Jan 23 11:41 .bashrc
-rw——- 1 joris joris 51 Jan 23 11:42 .history
-rw-r–r– 1 joris joris 171 Jan 23 11:41 .kshrc
-rw-r–r– 1 joris joris 375 Jan 23 11:41 .profile
-rw-r–r– 1 joris joris 153 Jan 23 11:41 .vimrc
[/code]
What’s even better, is that you can run multiple commands separated with a semi colon, like this:
[code highlight=”1″]
ssh my_server ‘ls -l /home/my_home_dir;whoami’
[/code]
And the best trick is this one, user input with an interactive command, sending input and output back and forth!
[code highlight=”1″]
ssh -t my_server ‘vi ~/.bash_profile’
[/code]

Find Java JRE location on Ubuntu Linux

Everybody knows that the java executable is located in /usr/bin/java , but what if you need the JDK / JRE location itself?
Just using “whereis” will not get you there, that will point you to the /usr/bin/java point.
joris@howlingmad: ~_011
 
So, let’s find out a but more about /usr/bin/java:

ls -l /usr/bin |grep java

joris@howlingmad: ~_012
 
Awesome, this will lead us somewhere, it’s a symlink to /etc/alternatives/java
So let’s do the same there:

ls -l /etc/alternatives/ |grep java

And we’ve hit the jackpot, among the lines here, there’s a bunch of lines pointing us to the JRE location:
joris@howlingmad: ~_013
 
As you can see in the screenshit, our java executable within the JRE location is:

/usr/lib/jvm/java-7-oracle/jre/bin/java

Which means the JRE location is:

/usr/lib/jvm/java-7-oracle/jre

 
 
 

CHere Bash Here without Admin Rights – CYGWIN

I wanted to add a “Bash Here” context entry when I right click on a directory:
Screenshot - 28-2-2014 , 13_01_20
This Bash Here would open that directory in Cygwin (mintty).
My current PC is fairly regulated, so without admin rights and thus I can’t use the CHere option which I would normally use.
I just created the registry keys via regedit and that works fine.
You can use two options:

  1. Import the registry file mentioned below
  2. Create the keys yourself

1. Import the registry file

Create a file called cygwin_bash_here.reg with the following contents:

Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Classes\Directory]
[HKEY_CURRENT_USER\Software\Classes\Directory\shell]
[HKEY_CURRENT_USER\Software\Classes\Directory\shell\bashhere]
@="Bash Here"
[HKEY_CURRENT_USER\Software\Classes\Directory\shell\bashhere\command]
@="C:\\cygwin\\bin\\mintty.exe"

Save the file and double click on it to import it to the registry.

2. Create the keys yourself

  1. Open the registry editor: Start – Run – “regedit”
  2. Open the following path: HKEY_CURRENT_USER\Software\Classes
  3. Create key: “Directory”
  4. Inside Directory, create key: “shell”
  5. Inside shell, create key: “bashhere”
  6. Inside bashhere double click on the standard key and insert text “Bash here” without quotes (This is the text which is displayed in the context menu.
  7. Inside bashhere, create key: “command”
  8. Inside command, double click on the standard key and insert text “C:\cygwin\bin\mintty.exe” without quotes. Important: This is the command which is run. Please change directory to your cygwin directory

Screenshot - 28-2-2014 , 13_16_02

Conclusion

Both options will create a context menu item which opens Cygwin at your current windows directory!

Cygwin – Your group is currently mkpasswd

This message occurs every time you start your freshly installed Cygwin when you’re logged in as a domain user.

Your group is currently "mkpasswd".  This indicates that your
gid is not in /etc/group and your uid is not in /etc/passwd.
The /etc/passwd (and possibly /etc/group) files should be rebuilt.
See the man pages for mkpasswd and mkgroup then, for example, run
mkpasswd -l [-d] >> /etc/passwd
mkgroup  -l [-d] >> /etc/group
Note that the -d switch is necessary for domain users.

Important: You need to install Cygwin with the user you are logged in with.
Tip
: Remove the word “setup” from the cygwin executable to be able to install it without administrator privileges. (e.g. setup-x64.exe should be renamed to cygwin-x64.exe)

  • mkpasswd -l only shows my local users, and not the domain user I’m logged in with, so that does not solve this.
  • mkpasswd -l -d get an enormous amount of users because it tries to replicate my whole organisation, which is not necessary.

We just need our current user ( mkpasswd -c ) to be sent to the /etc/passwd and /etc/group files, to do this, we use this command:

Solution:

mkpasswd -c >> /etc/passwd
mkgroup -c >> /etc/group

After that, our current account is added to both /etc/passwd and /etc/group and the annoying greeting message is gone!

SSH through a proxy to a remote server

I wanted to SSH into my home server from my workplace but I couldn’t reach it directly because of the way the network was set up.
As it turns out it is quite easy to do by using the corkscrew program.
Edit ~/.ssh/config and add the following lines:

Host home joris.his.homeserver.com
    Hostname joris.his.homeserver.com
    User joris
    ProxyCommand corkscrew proxyserveraddress proxyserverport %h %p

The most important part is the ProxyCommand, this lets your ssh client know that it should use corkscrew as a proxy to your host. %h means the host of your remote server, %p means the port of your remote server.