Menu

Log in

Sign up

From beginner to master of web design, coding, infrastructure operation, business development and marketing

  • COURSES
  • HTML & CSS Introduction
  • HTML & CSS Coding with AI
  • Linux Introduction
  • Docker Basics
  • Git & GitHub Introduction
  • JavaScript Coding with AI
  • Django Introduction
  • AWS Basics
  • Figma Introduction
  • SEO Tutorial for Beginners
  • SEO with AI
  • OTHERS
  • About
  • Terms of Service
  • Privacy Policy

© 2024 D-Libro. All Rights Reserved

Linux IntroductionChapter 5. Redirection, Pipe and Shell Script

uniq (Extract Unique Data Lines)

uniq (Extract Unique Data Lines)

Identifying Unique Data Lines with uniq Command

The uniq (UNIQue) command is used to delete adjacent duplicated lines. With the sort command, you can extract unique data lines.

uniq command syntax

To explain the cut command, we use the purchase_product.txt file created in the previous section.

Command Line - INPUT
cat purchase_product.txt
Command Line - RESPONSE
Apple
Orange
Apple
Apple
Bananas
Apple
Grapes
Orange
Bananas
Grapes
Grapes
Bananas
Bananas
Apple
Grapes

To delete adjacent duplicated lines, run the command below.

Command Line - INPUT
uniq purchase_product.txt

The result will be like the one below.

Command Line - RESPONSE
Apple
Orange
Apple
Bananas
Apple
Grapes
Orange
Bananas
Grapes
Bananas
Apple
Grapes

uniq command with sort

To extract unique data lines, connect to the sort command using a pipe like shown below

Command Line - INPUT
sort purchase_product.txt | uniq
Command Line - RESPONSE
Apple
Bananas
Grapes
Orange

-c option

If you want to show numbers of duplicated lines, use the -c option.

Command Line - INPUT
sort purchase_product.txt | uniq -c
Command Line - RESPONSE
      5 Apple
      4 Bananas
      4 Grapes
      2 Orange

You can also learn this topic offline. Click AmazonKindle.

More Topics to Explore

How to Do Linux Command Syntax

Linux Command Syntax

Chapter 5. Building and Sharing Docker Images

Chapter 5. Building and Sharing Docker Images

Understanding Firewalls in Linux

Firewall

Implementing SSL with Certbot for Django

SSL Setup – Certbot

Handling Special Characters in Linux

Special Characters and Escape Character

How to Do Linux Command Syntax

Linux Command Syntax

Chapter 5. Building and Sharing Docker Images

Chapter 5. Building and Sharing Docker Images

Understanding Firewalls in Linux

Firewall

Implementing SSL with Certbot for Django

SSL Setup – Certbot

Handling Special Characters in Linux

Special Characters and Escape Character

Tags:

Linux Command

Linux Introduction
Course Content

Chapter 1. Linux Basics

What Is OS?

CUI and GUI

Linux Distributions

Package Manager

Kernel and Shell

Current Working Directory

Linux Directory Structure

Absolute Path and Relative Path (Linux OS)

Linux Command Syntax

Special Characters and Escape Character

Chapter 2. Linux Key Commands

Setting Up Linux Environment on AWS

pwd (Print Working Directory)

cd (Change Directory)

ls (List Contents of Directory)

mkdir (Make Directory)

rmdir (Remove Directory)

touch (Create File)

rm (Remove File)

mv (Move File and Directory)

cp (Copy File and Directory)

cat (Display File Content)

sort (Sort File Contents)

grep (Global Regular Expression Print)

Regular Expression

find (Find File and Directory)

Wildcard

ln (Create Link to File and Directory)

Chapter 3. Vim Editor

What Is Vim and How to Launch It?

Normal, Insert and Visual Mode

Normal Mode (1) – Move Cursor

Normal Mode (2) – Delete

Normal Mode (3) – Copy and Paste

Normal Mode (4) – Undo and Redo

Normal Mode (5) – Search Phrase

Normal Mode (6) – Replace Phrase

Normal Mode (7) – Save and Exit

Insert Mode

Visual Mode

Chapter 4. User, Group and Permission

What Are User, Group And Permission in Linux?

Permission (Access Mode) by Owner Status

Superuser (Root User) vs. Normal User

sudo (Run Command with Superuser Privileges)

su (Switch User)

useradd (Add User)

passwd (Set Password)

userdel (Delete User)

Group – Primary Group and Secondary Group

groupadd (Add Group)

usermod (Modify User Account Information)

gpasswd (Add and Delete Users to Group)

groupdel (Delete Group)

chown (Change Owner of File and Directory)

chgrp (Change Group of File and Directory)

chmod (Change Access Mode)

chmod Command with Numbers

w and who (Check Current User Login Status)

id and groups (Check User ID and Group)

getent (Display User and Group Data)

Chapter 5. Redirection, Pipe and Shell Script

Standard Input Output and Redirection

Pipe (Combine Commands)

less (Display Content with Pager)

tr (Replace Characters)

cut (Extract Data Sections)

uniq (Extract Unique Data Lines)

Shell Script

echo (Echo input)

read (Read and Store Input)

Shell Variable and Environmental Variable

source (Execute Shell Script and Refresh Environmental Variables)

Chapter 6. Linux Commands for Command Management

history (Check Command History)

alias (Create Command Shortcuts)

man (Display Manual)

type, which and whereis (Display Command Information)

Package Manager Command

tree (Display Directory Tree)

Chapter 7. SSH Remote Connection

SSH (Secure Shell)

Locate .ssh Directory

SSH Remote Login (1) – Use Key Pair Generated by Server

SSH Remote Login (2) – Use Key Pair Generated by Client

SSH Config File

SSH Remote Login with Visual Studio Code

SCP (Secure Copy Protocol)

SCP with SSH Config File

SFTP (Secure File Transfer Protocol)

Other File Transfer Commands

Chapter 8. Linux Process Management

Process and Job

Foreground and Background Jobs

jobs and ps (Display Jobs and Processes)

Signals

Create, Stop and Terminate Jobs

Daemon Processes

What Is Service on Linux?

Systemd

Unit File

Systemctl Sub-Commands

Create Custom Unit and Start at Boot

Firewall

UFW (Uncomplicated Firewall)

Web Server

Launch Apache Web Server

Chapter 1. Linux Basics

What Is OS?

CUI and GUI

Linux Distributions

Package Manager

Kernel and Shell

Current Working Directory

Linux Directory Structure

Absolute Path and Relative Path (Linux OS)

Linux Command Syntax

Special Characters and Escape Character

Chapter 2. Linux Key Commands

Setting Up Linux Environment on AWS

pwd (Print Working Directory)

cd (Change Directory)

ls (List Contents of Directory)

mkdir (Make Directory)

rmdir (Remove Directory)

touch (Create File)

rm (Remove File)

mv (Move File and Directory)

cp (Copy File and Directory)

cat (Display File Content)

sort (Sort File Contents)

grep (Global Regular Expression Print)

Regular Expression

find (Find File and Directory)

Wildcard

ln (Create Link to File and Directory)

Chapter 3. Vim Editor

What Is Vim and How to Launch It?

Normal, Insert and Visual Mode

Normal Mode (1) – Move Cursor

Normal Mode (2) – Delete

Normal Mode (3) – Copy and Paste

Normal Mode (4) – Undo and Redo

Normal Mode (5) – Search Phrase

Normal Mode (6) – Replace Phrase

Normal Mode (7) – Save and Exit

Insert Mode

Visual Mode

Chapter 4. User, Group and Permission

What Are User, Group And Permission in Linux?

Permission (Access Mode) by Owner Status

Superuser (Root User) vs. Normal User

sudo (Run Command with Superuser Privileges)

su (Switch User)

useradd (Add User)

passwd (Set Password)

userdel (Delete User)

Group – Primary Group and Secondary Group

groupadd (Add Group)

usermod (Modify User Account Information)

gpasswd (Add and Delete Users to Group)

groupdel (Delete Group)

chown (Change Owner of File and Directory)

chgrp (Change Group of File and Directory)

chmod (Change Access Mode)

chmod Command with Numbers

w and who (Check Current User Login Status)

id and groups (Check User ID and Group)

getent (Display User and Group Data)

Chapter 5. Redirection, Pipe and Shell Script

Standard Input Output and Redirection

Pipe (Combine Commands)

less (Display Content with Pager)

tr (Replace Characters)

cut (Extract Data Sections)

uniq (Extract Unique Data Lines)

Shell Script

echo (Echo input)

read (Read and Store Input)

Shell Variable and Environmental Variable

source (Execute Shell Script and Refresh Environmental Variables)

Chapter 6. Linux Commands for Command Management

history (Check Command History)

alias (Create Command Shortcuts)

man (Display Manual)

type, which and whereis (Display Command Information)

Package Manager Command

tree (Display Directory Tree)

Chapter 7. SSH Remote Connection

SSH (Secure Shell)

Locate .ssh Directory

SSH Remote Login (1) – Use Key Pair Generated by Server

SSH Remote Login (2) – Use Key Pair Generated by Client

SSH Config File

SSH Remote Login with Visual Studio Code

SCP (Secure Copy Protocol)

SCP with SSH Config File

SFTP (Secure File Transfer Protocol)

Other File Transfer Commands

Chapter 8. Linux Process Management

Process and Job

Foreground and Background Jobs

jobs and ps (Display Jobs and Processes)

Signals

Create, Stop and Terminate Jobs

Daemon Processes

What Is Service on Linux?

Systemd

Unit File

Systemctl Sub-Commands

Create Custom Unit and Start at Boot

Firewall

UFW (Uncomplicated Firewall)

Web Server

Launch Apache Web Server