Saturday, July 9, 2016

JavaScript not evaluating all functions in condition

I’ve just been struck by a simple though nuance code optimization. I’ve met it probably thousands times. In fact, some style guides I’ve followed forbids to do what I’ve just done ;-)

First let me explain what we want to achive. We want to call 3 functions and early return if all 3 returned false. Does the below code look good for you?

if (!func0() && !func1() && !func2()) {
    return;
}

Unfortunately it’s wrong. The JavaScript execution engine will optimize it and if the func0 call returns true (negated by !) then the func1 and func2 won’t be called at all.

This phenomenon has a name and it’s called short-circut evaluation, quoting Wikipedia:

the second argument is executed or evaluated only if the first argument does not suffice to determine the value of the expression

This applies to third and further arguments. Thus the lesson is: never put function calls in logical statements, at least not functions you want guarantee to be executed.

Changed code:

var skip = !func0();
skip &= !func1();
skip &= !func2();
if (skip) {
    return;
}

I’m sure the world is full of this kind of bugs.

Tuesday, May 10, 2016

Postmortem culture

Culture

It happens, something goes wrong, the system goes down, it stops proceeding orders or serve ads or … situation becomes nervous.

I would say one of the main differences how organisation handles those critical situations is a pretty good indicator of it shape. I’ve seen a companies putting the employees under huge pressure during incidents and blaming after.

Then, I’ve worked for Google and… When you make mistake, the impact factor happens to be thousands QPS. You look at graph and see how some stats goes down or crazily spike. You rollback or do emergency release and it starts (ok, it’s more complex, but hey). You (in most cases more than you) are responsible for writing a postmortem.

What is a postmortem?

In short, it’s a document describing what happened. I know in many companies it looks different, many adopted it after some ex-Googler has joined them (can we call it Googleism? or Googleisation?). What is important in Google’ postmortems is it’s purpose and main pur aim of postmortem is: to learn and never repeat old mistakes.

That changes the perspective dramatically. Writing about your own fuckup is not trivial and writing in no-blame way is even harder. It’s not easy even for local stars (local genius theory). How to write postmortem? What to put in it? How it should look? Check out the links

What’s next?

The document usually should be created in next 24-48h after, unless the incident is very complex or not understood. Usually, the PM are open for comments, so everyone may ask a question for some details. When ready and the document went through some ‘peer-review’, it’s should be sent to all interested parties (mailing group, #slack, whatever) and it should be discoverable. Means, it should end up in bug tracker under the issue (because, you have an open issue about the outage, right?). It should be kept in some postmortems’ database. Thus, even if it’s not you who pushed a binary or write the code, you will learn. And in the future, you can always look for similar cases.

Wednesday, May 4, 2016

Read from file: Length-prefixed protocol buffers

File format

In short it’s binary encoded protobuf message prefixed with the size. For longer description please check Length-prefix framing for protocol buffers.

In most languages the file is read as stream. If you need to open and parse a file which holds lenght-prefixed protobuf messages in Python, you read internet and you find a code which read the whole file. It’s huge bottleneck. The hit in a performance between reading whole file then parsing byte after byte and using BufferedReader was in 1000x. Thus enjoy my little piece of code:

def ReadItm(fname, constructor, size_limit = 0):
    ''' Reads and parses a length prefixed protobuf messages from file. 
        The file MUST not be corrupted. The parsing is equivalent to parseDelimitedFrom.
    '''
    f = None
    if fname.endswith('.gzip'):
        f = gzip.open(fname, 'rb')
    else:
        f = open(fname, 'rb')
    reader = BufferedReader(f)
    bytes_read = 0
    while size_limit<=0 or bytes_read<size_limit:
        buffer = reader.peek(10)
        if len(buffer) == 0:
            break
        (size, position) = decoder._DecodeVarint(buffer, 0)
        reader.read(position)
        itm = constructor()
        itm.ParseFromString(reader.read(size))
        bytes_read = bytes_read + position + size
        yield itm
    f.close()

Thursday, April 7, 2016

Go: imagick image croping after rotation problem

I need to load image rotate it by some angle, then crop some part of it. Seems trivial, isn't?

I use: gopkg.in/gographics/imagick.v2/imagick package.

Load image
iwand := imagick.NewMagickWand()
defer iwand.Destroy()
if err := iwand.ReadImage(s); err != nil {
 log.Panicf("cannot open image %s", err)
}
Rotate it, fill new area with yellow
w := iwand.GetImageWidth()
h := iwand.GetImageHeight()
log.Printf("old size: %d,%d", w, h)
pwand := imagick.NewPixelWand()
pwand.SetColor("yellow")
if err := iwand.RotateImage(pwand, 45); err != nil {
 log.Panicf("problem with rotation: %s", err)
}
Calculate the position and size of the rectangle to cut
newW := calcSize(w, h)
newH := newW

w = iwand.GetImageWidth()
h = iwand.GetImageHeight()
log.Printf("new size: %d,%d", w, h)

x := int((w - newW) / 2)
y := int((h - newH) / 2)
Make the area red (with transparency), just for learning
dwand := imagick.NewDrawingWand()
pwand.SetColor("red")
pwand.SetOpacity(0.5)
dwand.SetFillColor(pwand)
dwand.Rectangle(float64(x), float64(y), float64(newW+uint(x)), float64(newH+uint(y)))
iwand.DrawImage(dwand)
Crop and save to file
log.Printf("w,h,x,y: %d,%d,%d,%d", newW, newH, x, y)
if err := iwand.CropImage(newW, newH, x, y); err != nil {
 log.Fatalf("problem with crop: %s", err)
}
iwand.WriteImage(d)

Input file
Original image (from: https://en.wikipedia.org/wiki/Linux)

Output file
Image rotated and cropped

Need to repage!
log.Printf("w,h,x,y: %d,%d,%d,%d", newW, newH, x, y)
iwand.ResetImagePage("")
if err := iwand.CropImage(newW, newH, x, y); err != nil {
 log.Fatalf("problem with crop: %s", err)
}
iwand.WriteImage(d)
Rotated and cropped - fixed
Full source: gist

Friday, March 11, 2016

Android developer road

So, I'm going through a quick Android developer course. The main target is to learn how to provide the best Android apps with minimal effort. Assumption is we all know Java ;-)

The start

A good starting point is:
http://developer.android.com/training/index.html

I don't recommend to read it all at the beginning. I would suggest:
Others as needed, Best Practices for Interaction and Engagement (http://developer.android.com/training/best-ux.html) also helps.

The above gives you a rough overview. The real stuff comes in API Guides (http://developer.android.com/guide/components/index.html). I do recommend to read the first three sections:
This gives you a high level overview of how the Android works.

The lifecycle

You basically should have it at your desk. You will use it often. It's been reported to have some inaccuracies but it's still better than the Google provided simplified version.



Codelabs

Google provides two interesting codelabs:
If you have to provide some early results quickly, skip above readings and just do the codelab.

Other must-to-know

Butterknife - http://jakewharton.github.io/butterknife/ - saves you lot of boilerplate with binding view elements to variables.
Android ContentProvider Generator - it will save you a lot of time on boilerplate code - https://github.com/BoD/android-contentprovider-generator
Mosby - http://hannesdorfmann.com/mosby - if you go with MVP this is must to use.
Dagger - dependency injection framework - http://google.github.io/dagger/users-guide

And when you consider which Android version to support, check:
Android stats Dashboard - gives an overview of the Android version distribution - http://developer.android.com/about/dashboards/index.html

Learning

Monday, February 8, 2016

No, because no

I need Math.Round from Go's standard library, poor me:


I know it's hard. But hey, most languages provide it.

Rob Pike:
Floating point numbers don't have digits.
The bar for being useful needs to be pretty high to be in the Go math package. We accepted most of what's in the C library but we're not going to take all the routines from other libraries. The Javascript round is a 1-liner: int(f+0.5). 

Wednesday, January 27, 2016

Angular-2.0.0.beta1

Http problem

I was getting an error:
error TS2305: Module '"angular2/http"' has no exported member 'HTTP_PROVIDERS'
may be caused by simply outdated angular2.d.ts installed through tsd.
DefinitelyTyped.org typings for Angular2 are outdated, tsd install angular2 gives old .d.ts which makes typescript unhappy. What’s the new approach or which angular.d.ts is appropriate?
When appropriate angular2 node package is installed everything works fine.

Duplicate properties

The other error I’ve encounter (Angular-2.0.0.beta.1):
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(6,14): error TS2300: Duplicate identifier 'PropertyKey'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(6,14): error TS2300: Duplicate identifier 'PropertyKey'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(9,5): error TS2300: Duplicate identifier 'done'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(10,5): error TS2300: Duplicate identifier 'value'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(248,5): error TS2300: Duplicate identifier 'EPSILON'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(283,5): error TS2300: Duplicate identifier 'MAX_SAFE_INTEGER'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(290,5): error TS2300: Duplicate identifier 'MIN_SAFE_INTEGER'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(346,5): error TS2300: Duplicate identifier 'flags'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(498,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(561,5): error TS2300: Duplicate identifier 'size'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(570,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(581,5): error TS2300: Duplicate identifier 'size'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(590,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(605,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(619,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(6,14): error TS2300: Duplicate identifier 'PropertyKey'.
ts/typings/es6-shim/es6-shim.d.ts(9,5): error TS2300: Duplicate identifier 'done'.
ts/typings/es6-shim/es6-shim.d.ts(10,5): error TS2300: Duplicate identifier 'value'.
ts/typings/es6-shim/es6-shim.d.ts(248,5): error TS2300: Duplicate identifier 'EPSILON'.
ts/typings/es6-shim/es6-shim.d.ts(283,5): error TS2300: Duplicate identifier 'MAX_SAFE_INTEGER'.
ts/typings/es6-shim/es6-shim.d.ts(290,5): error TS2300: Duplicate identifier 'MIN_SAFE_INTEGER'.
ts/typings/es6-shim/es6-shim.d.ts(346,5): error TS2300: Duplicate identifier 'flags'.
ts/typings/es6-shim/es6-shim.d.ts(498,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(561,5): error TS2300: Duplicate identifier 'size'.
ts/typings/es6-shim/es6-shim.d.ts(570,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(581,5): error TS2300: Duplicate identifier 'size'.
ts/typings/es6-shim/es6-shim.d.ts(590,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(605,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(619,5): error TS2300: Duplicate identifier 'prototype'.

I simply solved it by specifying the files in tsconfig.json explicitly through files. A better solution is to remove typings/es6-shim/.