How do I calculate sum of a column of numbers

I executed a command at a Linux machine, and it produced a long output.  I did the filtering, and now I have a column of numbers.  Now, how do I calculate sum of these numbers?  If you have awk available at your machine, here's how to do it.

$ your_long_list_of_command | awk '{ sum+=$1 } END { print sum}'

How to remove duplicate lines from a file

Our on-line publishing system has a text file that contains certain entries, one per line. Some entries were duplicate, and we wanted to remove them.

This can be done using sort and uniq commands.
sort /foo/bar | uniq > /new/bar

But we wanted to retain the order of lines, and so didn't want to sort the file. I found a solution using awk.
awk '!x[$0]++' /foo/bar > /new/bar

And how do I check if the file contains duplicate lines or not? The -d option of uniq command is helpful in this case.
sort /foo/bar | uniq -d

Why not to use awk when sed can do the job?

• Using awk instead of sed has the price of performance and size
• compared to sed and ed, awk takes a substantially longer time to load, and does its job at a considerably slower pace
• The real distinguishing point between sed and awk as a text processor is that awk is able to work with a persistent context, whereas capabilities of sed in this area are limited to non-existent. If you - for instance - would have to sum one field to a total you would do it with awk (it would be possible to do it with sed, but would be a nightmare - poorly suited tool for the job)