Regex and Grep cheat sheet

Cheat sheet based off the Udemy cysa+ course from Jason Dion – video 75 as i’m sure i’ll end up looking for it at some point in the future.

REGEX:

[] – Match a single instance of a chracter from a range such as a-z A-Z 0-9 or for all [a-zA-Z0-9]

[\s] – Match whitespace

[\d] – Match a digit

+ – Match one or more occurrences e.g. \d+-

*- Match zero or more occurrences e.g. \d*

? – Match one or none occureences e.g. \d?

{} – Match the number of times within the braces e.g. \d{3} finds 3 digits in a row or \d{7-10} matches 7,8,9 or 10 digits in a row

| – OR

^ – Only search at the start of a line

$ – Only search at the end of a line

GREP:

-F = search for a literal value, can use “” instead of -F

-r = recursive

-i = Ignore case sensitivity

-v = Find things which do not match

-w = Treat search strings as words (instead of parts of words)

-c = Show count of matches

-l = Return names of files containing matches

-L = Return names of files without matches

Advertisement

Hints and Tips for PythonChallenge level 3

This is a page of hints for the Pythonchallenge.com level 3 challenge

It does not contain the answer so you can use as many hints as you want but still have to put everything together yourself to complete it

Note: A lot of these challenges have multiple different ways of solving them, the hints here might not match to what you have found already.

Expand for hint 1

Anything interesting in the source code?

Expand for hint 2

3 big bodyguards? what could that mean?

Expand for hint 3

Can you open the text in python like in challenge 2?

Expand for hint 4

Can Python search for patterns in text?

Expand for hint 5

Maybe Regex can help?

Expand for hint 6

Exactly 3 bodyguards, not more.

Remove IP addresses from multiple log files

Log files can grow as large as you let them and if for any reason you need to redact the IP addresses they hold it’s often impractical to do it manually.

It can be automated using Notepad++ (A free text editor Notepad++ Download link) with some clever regex.

There are 2 different methods, one using simple regex which will do the job but will also remove some non-IP numbers which look similar in format to an IP address.

Quick and dirty method

Here are some demo log files with thousands of IP addresses inside:

Launch Notepad++ and select search >> find in files >>

The quick and dirty regex is : (\d{1,3}\.){3}\d{1,3}

set some replacement text or leave blank if you want to just delete the entries.

set the directory which contains your log files, and most importantly set the search mode to “regular expression”. It’s set to “Normal” by default.

When you press “Replace in Files” you’ll turn this:

Into :

As you can see it’s also gone through all of the log files and saved you a lot of manual time and effort:

Regex explained:

(\d{1,3}\.){3}\d{1,3}

(\d{1,3}\.) finds any set of numbers 1-3 digits in length with a full stop at the end.

{3} repeats the previous regex 3 times.

\d{1,3} is similar to the first part and searches for a set of numbers 1-3 digits in length but with no full stop at the end.

The reason you cannot simply use (\d{1,3}\.){4} is because the last octet of an IP address doesn’t end with a full stop.

The problem with this quick regex is that it will also pick up numbers such as:

999.999.999.999 which is not a valid IP. Depending on the contents of your log files this might not be a problem.

IP Specific regex

If the quick and dirty method doesn’t do exactly what you need you can use this much longer regex to specify the numbers which can appear in an IP address:

(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)

It will do mostly the same as the previous regex but is smart enough to not remove 999.999.999.999