Uniq
Encyclopedia
uniq is a Unix
utility which, when fed a text file, outputs the file with adjacent identical lines collapsed to one. It is a kind of filter program. Typically it is used after sort
. It can also output only the duplicate lines (with the -d option), or add the number of occurrences of each line (with the -c option).
An example: To see the list of lines in a file, sorted by the number of times each occurs:
Using uniq like this is common when building pipelines
in shell script
s.
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
utility which, when fed a text file, outputs the file with adjacent identical lines collapsed to one. It is a kind of filter program. Typically it is used after sort
Sort (Unix)
sort is a standard Unix command line program that prints the lines of its input or concatenation of all files listed in its argument list in sorted order. Sorting is done based on one or more sort keys extracted from each line of input. By default, the entire input is taken as sort key...
. It can also output only the duplicate lines (with the -d option), or add the number of occurrences of each line (with the -c option).
An example: To see the list of lines in a file, sorted by the number of times each occurs:
- sort file | uniq -c | sort -n
Using uniq like this is common when building pipelines
Pipeline (software)
In software engineering, a pipeline consists of a chain of processing elements , arranged so that the output of each element is the input of the next. Usually some amount of buffering is provided between consecutive elements...
in shell script
Shell script
A shell script is a script written for the shell, or command line interpreter, of an operating system. It is often considered a simple domain-specific programming language...
s.
Switches
- -u Print only lines which are not repeated (unique) in the original file
- -d Don't output lines that are not repeated in the input.
- -c Generate an output report in default style except that each line is preceded by a count of the number of times it occurred. If this option is specified, the -u and -d options are ignored if either or both are also present.
- -i Ignore case differences when comparing lines
- -f Ignore a number of fields in a line
- -s Skips a number of characters in a line
- -w Specifies the number of characters to compare in lines, after any characters and fields have been skipped
- --help Displays a help message
- --version Displays version number on stdout and exits.
External links
- uniq's LinuxLinuxLinux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
manpage - SourceForge UnxUtils – Port of several GNU utilities to Windows