Parallel programming notes

OpenMP

Introduction to OpenMP - Tim Mattson (Intel):
https://www.youtube.com/watch?v=jfQLD2AGSvc&list=PLLX-Q6B8xqZ8n8bwjGdzBJ25X2utwnoEG&index=0
OpenMP - Quick Reference Card:
http://openmp.org/mp-documents/OpenMP-4.0-C.pdf

CUDA

Measuring performance of CUDA operations
http://devblogs.nvidia.com/parallelforall/how-implement-performance-metrics-cuda-cc/
Monitoring GPU usage

$ nvidia-sli -l -lms 1000
+-----------------------------------------------------------------------------+
| Compute processes:                                               GPU Memory |
|  GPU       PID  Process name                                     Usage      |
|=============================================================================|
|    0            Not Supported                                               |
+-----------------------------------------------------------------------------+
Wed Sep 10 23:51:15 2014       
+------------------------------------------------------+                       
| NVIDIA-SMI 340.29     Driver Version: 340.29         |                       
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 770     Off  | 0000:01:00.0     N/A |                  N/A |
| 66%   80C    P0    N/A /  N/A |   1279MiB /  2047MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

..to be continued..

Using AppEngine remote_api in development environment

Note: This is a follow-up of a previous post Locally modifying Go package. As today (May 17, 2014) the changes described there are neccessary for a below code work.

We will modify: datastore_info.go provided as example of remote_api usage in App Engine Go SDK. One should follow the steps in Go SDK doc on remote_api to enable it.

Local client

The trivial function to sign in in a dev server as an admin is below.

func clientLocalLoginClient(host, email string) *http.Client {
 jar, err := cookiejar.New(nil)
 if err != nil {
  log.Fatalf("failed to make cookie jar: %v", err)
 }
 client := &http.Client{
  Jar: jar,
 }
 local_login_url := fmt.Sprintf("http://%s/_ah/login?email=%s&admin=True&action=Login&continue=", host, email)
 resp, err := client.Get(local_login_url)
 if err != nil {
  log.Fatalf("could not post login: %v", err)
 }
 defer resp.Body.Close()

 body, err := ioutil.ReadAll(resp.Body)
 if resp.StatusCode != http.StatusOK {
  log.Fatalf("unsuccessful request: status %d; body %q", resp.StatusCode, body)
 }
 if err != nil {
  log.Fatalf("unable to read response: %v", err)
 }

 m := regexp.MustCompile(`Logged in`).FindSubmatch(body)
 if m == nil {
  log.Fatalf("no auth code in response %q", body)
 }

 return client
}

Connecting to localhost instead a real app requires modifying a main to use the clientLocalLoginClient function if host address points to localhost:

 is_local := regexp.MustCompile(`.*(localhost|127\.0\.0\.1)`).MatchString(*host)
 if !is_local && *passwordFile == "" {
  log.Fatalf("Required flag: -password_file")
 }

 var client *http.Client
 if !is_local {
  p, err := ioutil.ReadFile(*passwordFile)
  if err != nil {
   log.Fatalf("Unable to read password from %q: %v", *passwordFile, err)
  }
  password := strings.TrimSpace(string(p))
  client = clientLoginClient(*host, *email, password)
 } else {
  client = clientLocalLoginClient(*host, *email)
 }

The full source code can be found here: https://gist.github.com/orian/3f74c6add4e4f572e108

The above code can be invoked as follow:

$ goapp run datastore_stats.go -host=localhost:8080 -email=test@example.com

Exporting data to sample app

Sample application with enabled remote_api in Go and data exporter can be found on GitHub github.com/orian/gae-go-remote-api-example. The prerequirement is a configured Google App Engine Go SDK.
Getting and starting the app:

cd workspace/go
git clone git@github.com:orian/gae-go-remote-api-example.git
cd gae-go-remote-api-example
goapp server

This starts app and logs 3 crucial info:

INFO     2014-05-17 21:30:27,120 api_server.py:171] Starting API server at: http://localhost:55542
INFO     2014-05-17 21:30:27,132 dispatcher.py:182] Starting module "default" running at: http://localhost:8080
INFO     2014-05-17 21:30:27,133 admin_server.py:117] Starting admin server at: http://localhost:8000

In another terminal one can:

cd workspace/go/gae-go-remote-api-example/examples
goapp run export_data.go --data_dir data/ -host localhost:8080 -email test@test.com

The terminal output should look similar to:

2014/05/17 23:47:07 appengine: not running under devappserver2; using some default configuration
2014/05/17 23:47:07 App ID "gae-go-boilerplate"
Skip: 
Visited: data/data_item_0.json
Visited: data/data_item_1.json
filepath.Walk() returned    # ironically this is good

This means that data from files data_item_0.json and data_item_1.json has been opened successfully and exported. One can check on admin panel of dev server: http://localhost:8000/datastore?kind=DataItem

Locally modifying Go package

Long story - short:
I'm playing with Google App Engine - Go version. I've tried to use one of the provided libraries and found out it doesn't work as I've expected.
A appengine/remote_api Client doesn't allow to connect to localhost and custom port, only default :80. I found a place in code responsible for handling localhost connection: https://github.com/golang/appengine/blob/a5bf4a208e232b1d3d1c972da47afe05b2c5faa5/remote_api/client.go#L46

    url := url.URL{
        Scheme: "https",
        Host: host,
        Path: "/_ah/remote_api",
    }
    if host == "localhost" {  // here's the reason
        url.Scheme = "http"
    }

then open terminal, go to directory where main go_appengine package is unpacked

cd ~/Downloads/software/go_appengine
find . -name remote_api
vim ./goroot/src/pkg/appengine/remote_api

and replace the above line with:

    if regexp.MustCompile(`^localhost(:\d{1,5})?$`).MatchString(host) {

Check it here: http://play.golang.org/p/fMogPEfgc8
There's one more thing one has to do, install modified package so it's used:

goapp install ./goroot/src/pkg/appengine/remote_api/

After this, if one run's goapp run my_super_tool.go it will use modified code. Pull request to original project.

GCC and vectorization

#include<ctime>
#include<iostream>
#include<memory>

using namespace std;

class Clock {
 public:
  Clock() : start_time_(std::clock()) {}

  std::clock_t Now() const { return std::clock() - start_time_; }

  double NowSeconds() const {
    return static_cast<double>(std::clock() - start_time_) / CLOCKS_PER_SEC;
  }

  void Reset() { start_time_ = std::clock(); }

 private:
  std::clock_t start_time_;
};

void Func(int* a, int *b, int* c, int size) {
  for(int i=0; i<size; ++i) {
    *a = *b * (*c);
    ++a; ++b; ++c;
  }
}

void FuncForSIMD(int* __restrict__ a, int* __restrict__ b, int* __restrict__ c,
                 int size) {
  for(int i=0; i<size; ++i) {
    *a = *b * (*c);
    ++a; ++b; ++c;
  }
}

int main() {
  constexpr int size = 1000000000;
  unique_ptr<int> a(new int[size]);
  unique_ptr<int> b(new int[size]);
  unique_ptr<int> c(new int[size]);
  Clock clock;
  Func(a.get(), b.get(), c.get(), size);
  cout << "time without simd: " << clock.Now() << endl;
  clock.Reset();
  FuncForSIMD(a.get(), b.get(), c.get(), size);
  cout << "time with simd: " << clock.Now() << endl;
  return 0;
}

To compile we need to run:

g++ -std=c++0x simd.cc -o simd -O3

The output on my machine looks as follow:

time without simd: 2160000
time with simd: 390000

So it gives 5.5x faster execution time.
Great explanation is given here:
Demystifying The Restrict Keyword - http://cellperformance.beyond3d.com/articles/2006/05/demystifying-the-restrict-keyword.html

Arduino IDE and Linux

After connecting Arduino UNO to Linux powered PC I've encountered two simple problems but which required from me to search for few minutes, less linuxy users may not found a solution, so here it goes.

Not existing device

Board at COM1 is not available

and during uploading

Arduino: 1.5.5 (Linux), Board: "Arduino Uno"

/home/pawel/software/arduino/arduino-1.5.5/hardware/tools/avr/bin/../lib/gcc/avr/4.3.2/../../../avr/lib/avr5/crtm328p.o: In function `__bad_interrupt':
../../../../crt1/gcrt1.S:193: undefined reference to `main'

One may need to create a symlink:

sudo ln -s /dev/ttyACM0 /dev/ttyCOM1

Or one should choose a correct port in Arduino IDE Tools->Port menu.

Permissions

avrdude: ser_open(): can't open device "/dev/ttyACM0": Permission deniedioctl("TIOCMGET"): Inappropriate ioctl for device

where /dev/ttyACM0 is my Arduino serial connection.

1. check user/group of device
   

   ls -l /dev/ttyACM0
   crw-rw---- 1 root dialout 166, 0 Jan 21 21:18 /dev/ttyACM0
   

2. add user 'pawel' to group 'dialout'
   

   sudo usermod -a -G dialout pawel