Read from file: Length-prefixed protocol buffers

File format

In short it’s binary encoded protobuf message prefixed with the size. For longer description please check Length-prefix framing for protocol buffers.

In most languages the file is read as stream. If you need to open and parse a file which holds lenght-prefixed protobuf messages in Python, you read internet and you find a code which read the whole file. It’s huge bottleneck. The hit in a performance between reading whole file then parsing byte after byte and using BufferedReader was in 1000x. Thus enjoy my little piece of code:

def ReadItm(fname, constructor, size_limit = 0):
    ''' Reads and parses a length prefixed protobuf messages from file. 
        The file MUST not be corrupted. The parsing is equivalent to parseDelimitedFrom.
    '''
    f = None
    if fname.endswith('.gzip'):
        f = gzip.open(fname, 'rb')
    else:
        f = open(fname, 'rb')
    reader = BufferedReader(f)
    bytes_read = 0
    while size_limit<=0 or bytes_read<size_limit:
        buffer = reader.peek(10)
        if len(buffer) == 0:
            break
        (size, position) = decoder._DecodeVarint(buffer, 0)
        reader.read(position)
        itm = constructor()
        itm.ParseFromString(reader.read(size))
        bytes_read = bytes_read + position + size
        yield itm
    f.close()

Go: imagick image croping after rotation problem

I need to load image rotate it by some angle, then crop some part of it. Seems trivial, isn't?

I use: gopkg.in/gographics/imagick.v2/imagick package.

Load image

iwand := imagick.NewMagickWand()
defer iwand.Destroy()
if err := iwand.ReadImage(s); err != nil {
 log.Panicf("cannot open image %s", err)
}

Rotate it, fill new area with yellow

w := iwand.GetImageWidth()
h := iwand.GetImageHeight()
log.Printf("old size: %d,%d", w, h)
pwand := imagick.NewPixelWand()
pwand.SetColor("yellow")
if err := iwand.RotateImage(pwand, 45); err != nil {
 log.Panicf("problem with rotation: %s", err)
}

Calculate the position and size of the rectangle to cut

newW := calcSize(w, h)
newH := newW

w = iwand.GetImageWidth()
h = iwand.GetImageHeight()
log.Printf("new size: %d,%d", w, h)

x := int((w - newW) / 2)
y := int((h - newH) / 2)

Make the area red (with transparency), just for learning

dwand := imagick.NewDrawingWand()
pwand.SetColor("red")
pwand.SetOpacity(0.5)
dwand.SetFillColor(pwand)
dwand.Rectangle(float64(x), float64(y), float64(newW+uint(x)), float64(newH+uint(y)))
iwand.DrawImage(dwand)

Crop and save to file

log.Printf("w,h,x,y: %d,%d,%d,%d", newW, newH, x, y)
if err := iwand.CropImage(newW, newH, x, y); err != nil {
 log.Fatalf("problem with crop: %s", err)
}
iwand.WriteImage(d)

Input file

Original image (from: https://en.wikipedia.org/wiki/Linux)

Output file

Image rotated and cropped

Need to repage!

log.Printf("w,h,x,y: %d,%d,%d,%d", newW, newH, x, y)
iwand.ResetImagePage("")
if err := iwand.CropImage(newW, newH, x, y); err != nil {
 log.Fatalf("problem with crop: %s", err)
}
iwand.WriteImage(d)

Rotated and cropped - fixed

Full source: gist

Android developer road

So, I'm going through a quick Android developer course. The main target is to learn how to provide the best Android apps with minimal effort. Assumption is we all know Java ;-)

The start

A good starting point is:
http://developer.android.com/training/index.html

I don't recommend to read it all at the beginning. I would suggest:

• Getting started (http://developer.android.com/training/index.html)
• Building apps with content sharing (http://developer.android.com/training/building-content-sharing.html)

Others as needed, Best Practices for Interaction and Engagement (http://developer.android.com/training/best-ux.html) also helps.

The above gives you a rough overview. The real stuff comes in API Guides (http://developer.android.com/guide/components/index.html). I do recommend to read the first three sections:

• Introduction (http://developer.android.com/guide/index.html)
• App Components (http://developer.android.com/guide/components/index.html)
• App Resources (http://developer.android.com/guide/topics/resources/index.html)

This gives you a high level overview of how the Android works.

The lifecycle

You basically should have it at your desk. You will use it often. It's been reported to have some inaccuracies but it's still better than the Google provided simplified version.

(check out: https://github.com/xxv/android-lifecycle)

Codelabs

Google provides two interesting codelabs:

• Build a Material Design App with the Android Design Support Library (https://codelabs.developers.google.com/codelabs/material-design-style/index.html)
• Android Testing Codelab (https://codelabs.developers.google.com/codelabs/android-testing/)

If you have to provide some early results quickly, skip above readings and just do the codelab.

Other must-to-know

Butterknife - http://jakewharton.github.io/butterknife/ - saves you lot of boilerplate with binding view elements to variables.

Android ContentProvider Generator - it will save you a lot of time on boilerplate code - https://github.com/BoD/android-contentprovider-generator
Mosby - http://hannesdorfmann.com/mosby - if you go with MVP this is must to use.
Dagger - dependency injection framework - http://google.github.io/dagger/users-guide

And when you consider which Android version to support, check:
Android stats Dashboard - gives an overview of the Android version distribution - http://developer.android.com/about/dashboards/index.html

Learning

Code Examples

https://github.com/mgrzechocinski/dagger2-example
https://github.com/cgruber/u2020/tree/dagger2
https://github.com/frogermcs/DaggerExample

Blog Posts

http://www.future-processing.pl/blog/dependency-injection-with-dagger-2/
http://fernandocejas.com/2015/04/11/tasting-dagger-2-on-android
http://konmik.github.io/snorkeling-with-dagger-2.html
http://siphon9.net/loune/2015/04/dagger-2-0-android-migration-tips/
http://blog.gouline.net/2015/05/04/dagger-2-even-sharper-less-square

No, because no

I need Math.Round from Go's standard library, poor me:

• https://github.com/golang/go/issues/4594
• https://groups.google.com/forum/#!searchin/golang-nuts/math.Round/golang-nuts/ITZV08gAugI/pNsL5FF4pLUJ

I know it's hard. But hey, most languages provide it.

Rob Pike:

Floating point numbers don't have digits.
The bar for being useful needs to be pretty high to be in the Go math package. We accepted most of what's in the C library but we're not going to take all the routines from other libraries. The Javascript round is a 1-liner: int(f+0.5). 

Angular-2.0.0.beta1

Http problem

I was getting an error:

error TS2305: Module '"angular2/http"' has no exported member 'HTTP_PROVIDERS'

may be caused by simply outdated angular2.d.ts installed through tsd.

DefinitelyTyped.org typings for Angular2 are outdated, tsd install angular2 gives old .d.ts which makes typescript unhappy. What’s the new approach or which angular.d.ts is appropriate?

When appropriate angular2 node package is installed everything works fine.

Duplicate properties

The other error I’ve encounter (Angular-2.0.0.beta.1):

ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(6,14): error TS2300: Duplicate identifier 'PropertyKey'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(6,14): error TS2300: Duplicate identifier 'PropertyKey'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(9,5): error TS2300: Duplicate identifier 'done'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(10,5): error TS2300: Duplicate identifier 'value'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(248,5): error TS2300: Duplicate identifier 'EPSILON'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(283,5): error TS2300: Duplicate identifier 'MAX_SAFE_INTEGER'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(290,5): error TS2300: Duplicate identifier 'MIN_SAFE_INTEGER'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(346,5): error TS2300: Duplicate identifier 'flags'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(498,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(561,5): error TS2300: Duplicate identifier 'size'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(570,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(581,5): error TS2300: Duplicate identifier 'size'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(590,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(605,5): error TS2300: Duplicate identifier 'prototype'.
ts/node_modules/angular2/typings/es6-shim/es6-shim.d.ts(619,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(6,14): error TS2300: Duplicate identifier 'PropertyKey'.
ts/typings/es6-shim/es6-shim.d.ts(9,5): error TS2300: Duplicate identifier 'done'.
ts/typings/es6-shim/es6-shim.d.ts(10,5): error TS2300: Duplicate identifier 'value'.
ts/typings/es6-shim/es6-shim.d.ts(248,5): error TS2300: Duplicate identifier 'EPSILON'.
ts/typings/es6-shim/es6-shim.d.ts(283,5): error TS2300: Duplicate identifier 'MAX_SAFE_INTEGER'.
ts/typings/es6-shim/es6-shim.d.ts(290,5): error TS2300: Duplicate identifier 'MIN_SAFE_INTEGER'.
ts/typings/es6-shim/es6-shim.d.ts(346,5): error TS2300: Duplicate identifier 'flags'.
ts/typings/es6-shim/es6-shim.d.ts(498,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(561,5): error TS2300: Duplicate identifier 'size'.
ts/typings/es6-shim/es6-shim.d.ts(570,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(581,5): error TS2300: Duplicate identifier 'size'.
ts/typings/es6-shim/es6-shim.d.ts(590,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(605,5): error TS2300: Duplicate identifier 'prototype'.
ts/typings/es6-shim/es6-shim.d.ts(619,5): error TS2300: Duplicate identifier 'prototype'.

I simply solved it by specifying the files in tsconfig.json explicitly through files. A better solution is to remove typings/es6-shim/.