What happened to turkiye.gov.tr

turkiye.gov.tr is our e-government gateway to many useful resources and hope this article finds its destinations.

Recently turkiye.gov.tr launched a new feature for viewing family trees. Great feature and work! There was a rush for many curious minds to find out about their ancestors. I guess turkiye.gov.tr wasn’t expecting too much traffic, so, things just went sideways in the most colossal way.

I am not familiar with the architecture of turkiye.gov.tr or the ancestry service but worked with many large-scale systems.

Below you will find 10 facts, I believe, caused outage for turkiye.gov.tr.

Fact #1: Big systems fail more often than small systems.
Big systems have more external and internal dependencies along with many moving parts. Therefore, big systems should have many defensive mechanisms in place.

Fact #2: Blocked threads are the number one cause of most failures.
Slow applications and hung threads are the most popular reasons of failures. These reasons lead to cascading failures and chain reactions. Blocked/hung threads can happen due to several reasons like deadlocks, starvation and live locks. Hung thread detection policies, timeouts, circuit breakers and bulkheads can prevent these failures.

Fact #3: Integration points are number one killer in any system.
Every integration point eventually fails, however a failure in the integration point shouldn’t take down the whole application. Cascading failures occur when problems in one integration point propagates. Failures in integrated services becomes your problem. It even become more serious if you are not prepared for it. Same as above, defensive programming, timeouts and circuit breakers can prevent serious failures.

Fact #4: Tight coupled systems fail more often than loose coupled systems.
Decoupling middleware is a good practice to enable loose coupling for integration. This principle is applicable and best practice for cloud native applications.

Fact #5: High traffic site’s Resource/Connections pools get drained very quickly.
Resource/Connection pool have limitations. They can run out of resources rapidly and application performance will start degrading.

Fact #6: Unbalanced capacities causes failures and scalability problems for applications.
If the capacities are not aligned, then you have a problem. Capacity and sizing should be planned accordingly.

Fact #7: Never trust a code you have no control over or you didn’t develop which can be a third partly library or a remote system developed by someone else.
The downstream application which is running a blocking code, can take your application down.

Fact #8: Slow applications gets more traffic.
When an application is slow, users hit re-load button or F5 many times to reach to the application which causes more traffic.

Fact #9: Fail fast all the time and retry gradually.
Exponential back off, Circuit breakers and timeouts should be embraced. User friendly error codes or default messages should be presented to the user until the stability is established.

Fact #10: Appreciate your hardware resources and utilize them wisely.
Don’t fall for CPU and Memory is CHEAP. This is not TRUE. Long running CPU cycles can cause contention which slows down your application and eventually it will fail. Paged or unfragmented memory causes slow seek times.

What happened reminds of dotcom bubble back in 2000s circa. Yahoo!, Altavista, Lycos and many internet giants back then faced similar problems and they started developing scalable platforms.

 

Git diff between commits

Many times you are interested in seeing the changes in a file between commits.  There are couple ways to do this.

#git diff commit_id..commit_id
something like:
#git diff abfa78e..b1317fc

This show the difference between two commits, that might be everything included in the commit.

If you are interested in the changes in a single file:

#git diff HEAD:file.name HEAD^:file.name
or
git diff HEAD:file.name HEAD^^:file.name

This will show the difference of a file between commits.

 

Git show contents of a commit

In order to see the contents of a commit:

#git show 4f62eae1b

This will print the content of the commit.

If you like to see the parents of a commit you can use ^ (caret sign). If you like see the parent of a parent commit use two of them ^^.

#git show 4f62eae1b^
This will show the parent of this commit.

#git show 4f62eae1b^^
This will show the parent of the parent commit.

It is also possible to use numbers.

#git show 4f62eae1b^2
This is equivalent to
#git show 4f62eae1b^^

Git show contents of current state or HEAD

In order to show the contents of the repository, you can use:

# git ls-tree HEAD
100644 blob 6d50415da79edca233601057a46bbb4d7c20494a HelloWorld.java
100644 blob 323fae03f4606ea9991df8befbb2fca795e648fa foobar

100644 indicates the permissions.
blob is the type of the object.
6d50415da79edca233601057a46bbb4d7c20494a is the object hash.
Then there comes the name of the File.

if you like to see the contents of the blob:

#git cat-file -p 6d50415
public class HelloWorld {

public static void main(String[] args) {
// Prints “Hello, World” to the terminal window.
System.out.println(“Hello, World”);
}

}

Git Object Model and Internals

Git is a powerful distributed version control system that tracks changes made to a set of files over time. Although it has been largely used for tracking software projects, it can be used to track any kind of content. In a few words, Git is a content tacker and a persistent map.

With Git you can:

  • Check the state of your project at any points in time
  • Show the differences among various states of the project
  • Split the project development into multiple independent lines, called “branches,” which can evolve separately
  • Periodically recombine branches in a process called “merging,” reconciling the changes made in two or more branches
  • Allow many people to work on a project simultaneously, sharing and combining their work as needed

There are 3 main concepts in Git.

Git has commits, snapshots and references. Below you will see Git Object model, it is more important for you to know and understand the Git object model rather than trying to learn all the commands.

Git repository is an object store which holds just four kinds of objects: blobstreescommits, and tags.

Objects:

  • Everything is an object.
  • Objects never change once created. They are immutable.
  • An object has
    • a type (Blob (File), Tree (Directory), commit (revision) or tag)
    • Size
    • Content
  • Objects are named with SHA1 hashes of contents
    • Very low chance of SHA1 collision
    • Long hashes: 40 digits long string.

Short hash 6 or 7 characters’ prefix is usually enough.

Blob:

Tree:

Commit:

This is essentially the Git Object Model.

Lets see it in action.

➜  mygit git init
Initialized empty Git repository in /Users/firat/mygit/.git/
➜  mygit git:(master) vim HelloWorld.java
➜  mygit git:(master) ✗ git add HelloWorld.java
➜  mygit git:(master) ✗ git commit -m "My first java program"
[master (root-commit) b6c1e40] My first java program
1 file changed, 8 insertions(+)
create mode 100644 HelloWorld.java
➜  mygit git:(master)

So far so good.

➜  mygit git:(master) git log
commit b6c1e4087e58c111693df0e3ab5053d6c2567a65
Author: firat <[email protected]>
Date:   Wed Dec 13 22:46:59 2017 +0300

My first java program

This is my first commit. Lets see what is inside this commit:

git cat-file helps to see the content of the commit.

➜  mygit git:(master) git cat-file -p b6c1e40
tree ca2cf428672bef33f1c8d182e3939dcb95fac84f
author firat <[email protected]> 1513194419 +0300
committer firat <[email protected]> 1513194419 +0300

My first java program

Above is my commit. It prints the information about this particular commit. Author, committer, Tree and commit message is included within this commit.


➜  mygit git:(master) git cat-file -p ca2cf428
100644 blob 6d50415da79edca233601057a46bbb4d7c20494a	HelloWorld.java

You can look what is inside the tree with `git cat-file` command recursively. This indicates that this tree has a blob, 100644 indicates the permission of the blob, which can come handy at specific situations.

Now lets look what is inside this blob.

➜  mygit git:(master) git cat-file -p ca2cf428
100644 blob 6d50415da79edca233601057a46bbb4d7c20494a	HelloWorld.java
➜  mygit git:(master) git cat-file -p 6d50415
public class HelloWorld {

public static void main(String[] args) {
// Prints "Hello, World" to the terminal window.
System.out.println("Hello, World");
}
}

This object which is a blob contains my HelloWorld.java content and nothing else.

In a commit with one file, you create at least 3 objects, you can find out how many objects within your repo as follows:

➜ mygit git:(master) git count-objects
3 objects, 12 kilobytes

You can play around with git cat-file and it’s options, traverse your repository and you will have better understanding of how git internal works. For example, you can have multiple directories, you can create tags, branches, then merge them, have multiple commits see the parent of the commits.

Git view files in another branch without checkout

If you like to view a file in another branch without (without checking it out) you can do this:

git show branch:file

So if you are in a branch called Feature_fancy and there is a file called fancy_controller.js:

git show Feature_fancy:fancy_controller.js

This will show the content of the file without having to check out this branch.

Git remove untracked files

If you have bunch of untracked files that you haven’t staged and they are sitting around, you can clean them all in once. While running git clean you have 3 options.

-n option does a dry run and shows you which files will be removed.

#git clean -n

-i option asks you interactively what to do:

#git clean -i

The last option is with -f option which forces removal of the unstaged files.

#git clean -f

That is about it.

Git Amend your last commit

Git has great features. One of them is to amend your previous commit. If you left something out on your last commit which can a file, or a change in a file that you just committed. Git amend can easily fix your problem.

You need to stage your changes with git add first.

#git add .

Then commit it again with :

#git commit –amend

You can also use –m option to overwrite the previous commit message.

#git commit –amend -m “your new message”

Once you are done you can check the log with:

#git log –stat

Git amend is pretty useful to recover from incomplete commits.