Monthly Archives: December 2016

3 Useful GUI and Terminal Based Linux Disk Scanning Tools


There mainly two reasons for scanning a computer hard disk: one is to examine it for filesystem inconsistencies or errors that can result from persistent system crashes, improper closure of critical system software and more significantly by destructive programs (such as malware, viruses etc).

And another is to analyze its physical condition, where we can check a hard disk for bad sectorsresulting from physical damage on the disk surface or failed memory transistor.

Read more at Tecmint

Click Here!

Remote Logging With Syslog, Part 4: Log Rotation


As you’ll recall, we’ve been taking a deep dive into the rsyslog tool for logging — we presented an overview in the first article, took a look at the main config file in the second article, and examined some logfile rules and sample configurations in the third. In this final article in the series, I’ll look at the /etc/rsyslog.d/www-rsyslog.chrisbinnie.tld.conf and discuss some important networking considerations.

Have a look at Listing 1, which shows the entirety of our /etc/rsyslog.d/www-rsyslog.chrisbinnie.tld.conf file.

$ModLoad imfile

$InputFileName /var/log/apache2/access_log

$InputFileTag apache.access:

$InputFileStateFile /tmp/apache_state_file

$InputFileSeverity info

$InputFileFacility local3

$InputRunFileMonitor

local3.* @@10.10.1.3:514

Listing 1: Contents of our remote application’s config using the trusty “local3” within the /etc/rsyslog.d/www-rsyslog.chrisbinnie.tld.conf file.

The key lines to focus on in this listing, starting from the top, are as follows.

We need to load up a module using $ModLoad. Step forward the outstanding “imfile” module, which has the magical ability to convert any normal text content into a rsyslog message. The manual says it will gratefully consume any printable characters that have a line feed (LF) at the end of each line to break up the otherwise monotonous content. Pretty clever, I’m sure you’ll agree.

The next important line is obvious. The line starting $InputFileName tells rsyslog which log file you’re interested in sending off to your remote logging server. The following line helps classify the log type with a “Tag” (which if you have multiple servers of the same application type sending logs to one remote server you might alter slightly per server to apache-apache-ww1: etc). Ignore the $InputFileStateFile log file for now and skim through the remaining lines.

We are collecting an “info” level of logging detail and pushing that out to the user-configurable “local3” facility and onto the IP Address “10.10.1.3”. The emoji that you can see — the two @ signs — stands for TCP. Only one @ sign would signify transfers via the UDP networking protocol.

Can Anybody Hear Me?

What about our remote rsyslog server’s config?

Have a quick look at the two lines below, which will sit in your remote rsyslog server’s /etc/rsyslog.conf file. For the sake of clarity, this is the recipient rsyslog server, the one that’ll receive all the logs from multiple places. Unless you’ve lost control of your senses, these two lines are easy to follow:

$ModLoad imtcp

$InputTCPServerRun 514

Incidentally, if an IP address is omitted (it can be explictly stated using something like $TCPServerAddress 10.10.10.10), then rsyslog will attempt to open up all IP addresses on the port in question.

You might be pleasantly surprised at how easy it is to finish off the remote rsyslog server config. We use something called “templates” in rsyslog. They are powerful, extensible, and worth reading about in more detail.

At the foot of our /etc/rsyslog.conf file, we simply add these lines:

$template ChrisBinnie, “/var/log/%HOSTNAME%/%PROGRAMNAME%.log”

*.*   ?ChrisBinnie

I’ve just used an arbitrary template reference in this case, my name, for ease of distinction. You will need to restart the service after making these changes.

We can see that we’re going to drop our logs into a directory off of /var/log, which has a hostname and then the application name. Appended to the end, the “.log” makes sense of the resulting filename. You can see which facilities and priorities are being added to this log file, in this case, “all” and “all” — thanks to the asterisks.

To TCP Or Not To TCP

You might need to spend a second thinking back over some of the remote logging considerations I mentioned, such as network congestion, if you’re pushing lots of logging data across your LAN. I mention this because (as we saw above) the clever rsyslog can use both TCP and UDP for pushing your logs around. TCP is the option best suited to most scenarios, thanks to its ability to error-check against network failures. It also doesn’t require an additional plugin to be loaded up, because it’s built into rsyslog; the reverse is true for the UDP protocol.

There are two minor connection points to note here. First, avoid using hostnames via DNS. Use an IP address for greater reliability (CNAMEs are somewhere in the middle, if you change machines around every now and again). Second, as with all things that might need debugged at some point, you should try to use explicit port numbers on both server and client ends so that there’s no ambiguity introduced. Incidentally, without a port explicitly specified, both protocols default to 514.

Little Dutch Boy

Note that if you’re using properly configured systems, you might need to punch a hole in the dike. Looking at the example below, clearly you need to alter the port number after –dport to your port number of choice. You can then save the setting to make it persistent with something like /sbin/service iptables save or whatever your distribution prefers.

If you need to allow access using the default TCP option and you’re using iptables, you can use this command:

# iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 514 -j ACCEPT

And, for good, old UDP, use:

# iptables -A INPUT -p udp -m state --state NEW -m udp --dport 514 -j ACCEPT

Stale Bread

Back to the dreaded gotcha. So that we’re clear: On the client or sender machine, we make a little fix to our log rotation bug/problem. Not the remote rsyslog server.

At the risk of repeating myself, there may be better ways of fixing the log rotation problem (with a version upgrade, for example); however, read on to see what fixed it for me. Remember that the issue is that after “logrotate” had run, the remote logging stopped. The solution was a simple script set to run via a cron job after “logrotate” had run in the middle of the night.

There is some method to the madness of my script. By running it, we effectively insist that after a log rotation has taken place the faithful rsyslog pays attention to its “state” file. Let’s peek into an example now:

<Obj:1:strm:1:

+iCurrFNum:2:1:1:

+pszFName:1:25:/var/log/apache2/access_log:

+iMaxFiles:2:1:0:

+bDeleteOnClose:2:1:0:

+sType:2:1:2:

+tOperationsMode:2:1:1:

+tOpenMode:2:3:384:

+iCurrOffs:2:7:1142811:

>End

.

Listing 2: Our rsyslog “state” file example.

We shouldn’t profess to understanding what’s going on in Listing 2, I would hazard a guess that rsyslog is counting lines it’s processed — among other things, to keep it operating correctly. Let’s promptly move on to Listing 3. This is the cron job and script solution that fixed the issue for me.


#!/bin/bash


#

# Deletes stale rsyslog “state” files, appends a timestamp to the new filename in /tmp & restarts rsyslog.

#

# After rsyslog restart the remote logging node should catch up any missed logs fairly quickly.

#


# Declare var

timestamp=$(date +%s)


# Delete all “state” files in /tmp

/bin/rm -f /tmp/apache_state_file*


# Edit rsyslog file which sends data remotely in order to show newly named “state” file

/bin/sed -i.bak 's/apache_state_file-[0-9]*/apache_state_file-'"$timestamp"'/g' /etc/rsyslog.d/www-syslog.chrisbinnie.tld.conf


# Apply changes to “state” file in use

/sbin/service rsyslog restart

Listing 3: Our quick-fix script to get log rotations to speak to rsyslog satisfactorily after “logrotate” has finished doing its business.

If you read the comments at the top of the script then all going well the scripts raison d’etre should make sense. This script is run sometime around 5am after the excellent “logrotate” has finished its sometimes lengthy business. There’s no point in running the script during or before “logrotate” has finished its run (during testing my remote logging still failed).

The small script works like this (you probably need to run it twice initially to clean up /tmp filenames, which should be harmless if you do it manually). It deletes the old “state” file upon which rsyslog relies, works out the current time, and appends that time to the end of the “state” filename.

As a result, the /tmp directory ends up having one or two files that look like this apache_state_file-1321009871. It then backs up the existing remote logging config file and changes the “state” file that it references. Finally, a super-quick service restart means that the remote logging data starts up again and the other end (the remote rsyslog server) catches up with any missed logs in a whizz-pop-bang if there’s lots of data.

My experience is that if you tail the recipient log after running this script (or just performing a restart), you’ll see the catchup taking place super-speedily. In case you’re wondering, I found that sometimes a service restart didn’t pick up the pieces properly but altering the “state” file it referenced was successful without fail. Your mileage might vary of course.

As mentioned, I’m a little suspicious of an old version issue that I needed to fix with this script. In the current version, you can see there is some additional information about the current version’s “state” file. I hope my solution gives you some pointers and helps you out if you encounter a similar scenario, however.

I Can’t Hear You

If your already exceptionally noisy log file /var/log/messages begins to fill up with your application’s logs, too, then here’s another little life-saver. The workaround is simple, you just need a service restart, after applying this ;local3.none addition to the relevant destination log file line in /etc/rsyslog.conf:

*.*;auth,authpriv.none;local3.none  -/var/log/messages

No doubt you get the idea, this disables “local3” logging locally.

End Of Watch

I should have probably put more emphasis on the how configurable and extensible rsyslog is at the start. It would be remiss not to point you at the extensive, well-maintained, documentation, which is all linked to from the application’s main website. There are modules and templates galore to explore in high levels of detail. And, as well as user examples on the main site, there’s an excellent wiki with some trickier configuration examples. If you’re eager for even more, you can check out this list of modules.

Now that I’ve presented the solution to a working remote rsyslog server, even with the challenges that log rotation throws into the mix, I hope you’ll think more about your logging infrastructure. I had originally used a Syslog server, back in the day, to pick up salient events from Cisco routers. So universal are the logging formats supported by rsyslog that you can also connect them to all sorts of devices, such as load balancers and other proprietary devices. You are far from being limited to picking up logs from other Unix-type devices and instead are blessed with the ability to collect logging event information from all over the place.

I hope you enjoy putting your new logging knowledge to creative and productive use.

Read the other articles in this series:

Remote Logging With Syslog, Part 1: The Basics

Remote Logging With Syslog, Part 2: Main Config File

Remote Logging With Syslog, Part 3: Logfile Rules

Chris Binnie is a Technical Consultant with 20 years of Linux experience and a writer for Linux Magazine and Admin Magazine. His new book Linux Server Security: Hack and Defend teaches you how to launch sophisticated attacks, make your servers invisible and crack complex passwords.

Advance your career in system administration! Check out the Essentials of System Administration course from The Linux Foundation.

Step By Step Ubuntu 16.10 (Yakkety Yak) LAMP Server Setup


Sponsored Link

In around 15 minutes, the time it takes to install Ubuntu Server Edition, you can have a LAMP (Linux, Apache, MySQL and PHP) server up and ready to go. This feature, exclusive to Ubuntu Server Edition, is available at the time of installation.The LAMP option means you don’t have to install and integrate each of the four separate LAMP components, a process which can take hours and requires someone who is skilled in the installation and configuration of the individual applications. Instead, you get increased security, reduced time-to-install, and reduced risk of misconfiguration, all of which results in a lower cost of ownership.Currently this installation provide PostgreSQL database, Mail Server, Open SSH Server,Samba File Server, Print Server, Tomcat Java Server,Virtual Machine Host,Manual Package selection,LAMP and DNS options for pre-configured installations, easing the deployment of common server configurations.

Ubuntu LAMP server Install the following Versions

Ubuntu 16.10 (Yakkety Yak)
Apache 2.4.18
Mysql 5.7.16
PHP 7.0.8

First you need to download server version of Ubuntu from here after that create a CD and start booting with the CD Once it starts booting you should see the following screen in this you need to select your language and press enter

1

Now you need to select “Install Ubuntu Server” and press enter

2

Select your language and press enter

Select your location and press enter

If you want to try to have your keyboard layout detected by pressing a series of keys you need to select yes option.If you want to choose from a list click no

Select Origin of keyboard and press enter

Select keyboard layout and press enter

Detecting hardware to find CD-ROM Drivers in progress

Loading additional components in progress

Configures the network with DHCP if there is a DHCP server in your network

Enter your server Hostname

You need enter the Full name of the user you want to create for your server in this example i have created ubuntu user select continue and press enter

12

Enter your user account name here

13

Entered the password for test user select continue and press enter

9

Confirm password for test user

10

If you want to configure encrypted private directory select yes otherwise no and press enter

Configuring clock option

Detecting disks and all other hardware

You have to partition your hard disk in this example i have selected use entire disk option.If you want to do manually you can choose manual option and press enter.Make sure you have swap partition in place

Warning message about data lost on your hard disk

Guided Partitioning

Write the changes to disk here you need to select yes and press enter

Creating ext4 file system in progress

Installing system in progress

Configuring the package manager select continue and press enter

Configuring package mirror this will be related to your country option

Select and install software in progress

Select how do you want to configure automatic update press enter

Now it will start Installing software and here you need to select the server options here i have selected as openssh server and LAMP server installation.If you want to select each package separately select “Manual package selection” option

11

At the time of software installation it will prompt for mysql server root password enter root password of your choice and select continue

12

Confirm mysql server root password and select continue

13

 

Software installation is in progress

Installing GRUB Boot loader in progress

Finishing installation in Progress

Installation complete message here you need to remove your CD select continue and press enter it will reboot your server

After rebooting your server it will prompt for username and password once you logged in you should see similar to the following screen

This will complete the Ubuntu 16.10 (Yakkety Yak) LAMP Server Installation and your server is ready for installing applications which supports Apache2,Mysql and PHP7.

Configuring Static ip address in Ubuntu server

 

Ubuntu installer has configured our system to get its network settings via DHCP, Now we will change that to a static IP address for this you need to edit

Edit /etc/network/interfaces and enter your ip address details (in this example setup I will use the IP address 10.0.2.10):

sudo nano /etc/network/interfaces

and enter the following save the file and exit (In vi, ESC, then ZZ to save and exit)

# The primary network interface

auto enp0s3
iface enp0s3 inet static
address 10.0.2.10
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.2.255
gateway 10.0.2.1

Now you need to restart your network services using the following command

sudo service networking restart

You need to setup manually DNS servers in resolv.conf file when you are not using DHCP.

sudo vi /etc/resolv.conf

You need to add look something like this
search domain.com
nameserver xxx.xxx.xxx.xxx

Sponsored Link



Related posts

How to Build a Ceph Distributed Storage Cluster on CentOS 7


Ceph is a widely used open source storage platform. It provides high performance, reliability, and scalability. The Ceph free distributed storage system provides an interface for object, block, and file-level storage. Ceph is build to provide a distributed storage system without a single point of failure. In this tutorial, I will guide you to install and build a Ceph cluster on CentOS 7.

Read the complete article at HowToForge.

Click Here!

Bluestar Linux: A Beautiful Take on KDE and a User-Friendly Arch-Based Distribution


Have you ever wanted a combination of Arch Linux and KDE but always seemed to get stumped at the Arch Linux portion of the combination? If that’s you, your days of being left out in the Arch Linux/KDE cold are over. Why? Bluestar Linux.

This new(ish) kid on the block allows you to enjoy Arch Linux without having to jump through all the usual hoops of setting the distribution up manually, plus it offers a rather unique take on KDE, one that had me instantly nodding my head in agreement with their layout. In fact, what Bluestar did with KDE makes so much sense, it has me wondering why this isn’t the default layout of the “K” Desktop Environment (more on this in a bit).

The necessary specs

If you plan on running Bluestar Linux, the only specs you really need to be concerned about are the following:

The RAM requirements seem a bit high, but this is really a KDE recommendation. If you glance at the KDE system requirements, you see the Bluestar minimum makes sense.

  • Processor: Minimum 1 GHz (x86) Recommended Better than 1 GHz (x86)

  • Memory: Minimum 512 MB Recommended 1 GB

  • Hard drive capacity Minimum 4 GB Recommended 10 GB

NOTE: After a failed install on VirtualBox, having given 8GB to the hard drive and 2GB of RAM, I upped the system to 16GB of space and 3GB of RAM, and the installation succeeded. So these aren’t your grandfather’s Linux requirements.

First look

When you first boot up the live Bluestar image, what will immediately strike you is the layout of the KDE desktop (Figure 1).

Instead of the usual bottom panel, the Bluestar desktop opts for a dock at the bottom of the screen and an upper panel. The dock is fairly a straightforward implementation of Cairo dock. The upper panel is a very nicely designed take on the standard KDE panel that is split into three parts. The first section of the panel resides in the upper left corner of the desktop, where you’ll find the applications menu, three different folder shortcuts (one for ~/, one for shortcuts, and one for stack…where you can drag and drop items for quick access).

The second section of the panel resides in the center of the top edge of your screen and is set to autohide. Hover your cursor over this area to reveal the hidden section that contains a few handy widgets and the system tray/notification area (Figure 2).

The items in the autohide section of the panel include (from left to right):

  • Clock

  • Virtual keyboard widget

  • Calculator widget

  • System load widget

  • Language selector

  • Clipboard

  • Networking widget

  • Notification area

Finally, the third section of the upper panel is dedicated to the pager, another system load widget, and the logout/shutdown button.

On the desktop proper, you’ll find three default widgets: A folder widget, a clock widget, and a disk space widget. These widgets are locked but can be removed by right-clicking one of them and selecting Default Desktop Options > Unlock Widgets. Once you’ve unlocked the widgets, you can then hover your mouse over one of the widgets to reveal the hidden bar that allows you to move, resize, customize, and delete each (Figure 3).

Finally, click on the Applications menu to reveal everything installed on your desktop. You’ll be surprised at how much is included. This isn’t your bare-bones distribution. Within Bluestar, you’ll find all the productivity tools you need (including KOrganizer, LibreOffice, Okular, Sieve Editor), Development tools (including GHex, KAppTemplate, KUIViewer, Qt Assistant, QT Designer), Graphics apps (including GIMP, Gwenview, and KolourPaint), more Internet tools than you thought you might need (including Akregator, Avahi SSH Server Browser, Blogilo, Chromium, Cloud Storage Manager, Dropbox, FileZilla, Firefox, KGet, MEGASync, Pidgin, and QTransmission), multimedia tools to please just about anyone (including AMZ Downloader, Amarok, Dragon Player, GNOME Alsa Mixer, JuK, Kdenlive, KsCD, SMPlayer, SMTube, VLC), and a very large collection of utilities (such as Ark, Caffeine, ClipIt, Disks, Filelight, Galculator, Jovie, KGpg, Kate, SuperKaramba, and Sweeper).

Another nice touch found in Bluestar is all the codecs you’ll need to play your favorite media formats. As well, streaming sites (such as Youtube and Netflix) work out of the box.

Installation

The first thing you must do is download the Bluestar ISO. Once you boot up the live disk, click on the Application menu and then select Bluestar Linux Graphical Installer (Figure 4). This will walk you through the fairly easy installation wizard.

The installation of Bluestar Linux is a breath of fresh air for those who have wanted to give Arch Linux a try but had no desire to struggle through the challenge of setting up the system manually. Installing Bluestar is about as standard as installing any modern Linux distribution.

Post install

Once you’ve installed Bluestar Linux, you’ll be dealing with the pacman package management system. This means you’ll need to get up to speed with a few basics. First off, because this is a rolling release distribution, you’ll want to make sure your package databases are in sync. Do this by opening up Konsole (from Applications > System) and issuing the command sudo pacman -Sy. Type your sudo password, hit Enter, and your databases will sync.

Next, I recommend optimizing your pacman databases. Do this with the command sudo pacman-optimize && sync.

Finally, we’ll update the installed packages with the command sudo pacman -Su.

If you’re not a fan of the command line (which you should be if you’re running Arch Linux), you can always opt for the GUI front-end for pacman, PacmanXG. This is an incredibly powerful graphical tool that allows you to take on all the tasks associated with pacman (Figure 5).

Once all your packages are updated, you are ready to start playing around with Bluestar’s take on the KDE desktop and enjoying your Arch Linux-powered distribution.

Arch Linux the way it should be

I’ve played around with Arch Linux enough to recognize that Bluestar Linux has done something quite wonderful…made Arch accessible to anyone. This isn’t a distribution of Linux that you will struggle to get up to speed with. Bluestar is as user-friendly a take on Arch Linux as you will find. If you’ve been wanting to dive into the Arch waters, now is the time and Bluestar is your best bet.

Advance your career in system administration! Check out the Essentials of System Administration course from The Linux Foundation.