Thursday, December 17, 2015

Google Identity Toolkit for Android - weird compilation error

I’m hacking tiny Android app, one of the libraries I use is Google Identity Toolkit (it’s super easy to integrate). A documentation states that a Play Services should be added as dependency:
compile 'com.google.android.gms:play-services:8.3.0'
For any non trivial application it’s going to be a problem resulting in below error (or similar):
Error:Execution failed for task ':app:transformClassesWithDexForDebug'.
> com.android.build.api.transform.TransformException: com.android.ide.common.process.ProcessException: org.gradle.process.internal.ExecException: Process 'command '/usr/lib/jvm/java-8-oracle/bin/java'' finished with non-zero exit value 2
Search on Internet points to StackOverflow question: com.android.build.transform.api.TransformException. Apparently the bugs are related. The exit return code 2 is meaningless unless one has any clues. Anyway, the problem can be solved by replacing the dependency by:
compile 'com.google.android.gms:play-services-plus:8.3.0'
compile 'com.google.android.gms:play-services-auth:8.3.0'
I’ve opened an issue on Google Identity Toolkit - Android sample project asking for more documentation.
The error is related to a constraint of Android Dalvik runtime of to 65K Reference Limit. There are two solutions for the problem: decreasing the references (usually by chosing smaller dependencies, in our case subpackages instead of the full Play Services) and using so called multidex (some 3rd party blog).

Monday, October 19, 2015

Google Analytics doesn't work a.k.a. name collision in JavaScript

Today is another great day I’ve encounter a trivial bug hard to catch. It goes as follow. I went to Google Analytics to setup a new page. I’ve followed the instruction and I’ve copied into a source:

<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-AAAAAAAA-X', 'auto');
  ga('send', 'pageview');

</script>

and it didn’t work ;-)

First I’ve thought is my mistake, i’ve put a code just before </html>, but after trying </head> it wasn’t any better.

I’ve tried to copy paste the code to developer console and tadah, it works! Here I should mention that I use both Google Closure Library and Google Closure Compiler on my site. After thinking and poking the code for a moment I’ve made an educated guess:

Closure compiler compiled code is colliding a name with Google Analytics!

Took me 1 minute to verify and yes, I’ve got it.

The solution was straight:

<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','//www.google-analytics.com/analytics.js','dzialaj');

  dzialaj('create', 'UA-AAAAAAAA-X', 'auto');
  dzialaj('send', 'pageview');

</script>

Just for quick explanation, the string I’ve changed is a parameter to function which sets up a Google Analytics code on a website, it is a name that should be used as a Analytics main function and may be anything one wants.

Thursday, October 8, 2015

Save yourself and pass the context by

One of extending packages of Go is golang.org/x/net/context package. The idea comes form task oriented programming. You get a task to execute but want to attach some data to it. It appears to be useful when programing a RPC or web server solutions. In such cases your listener creates a context object and pass it to handlers. The handlers keep it and use it when appropriate.

This idea guided most Google App Engine APIs for Go. Now they’ve introduced a Managed VM’s which are something between App Engine and Compute Engine. The refactored libraries can be found here: google.golang.org/appengine.

The standard way of logging in App Engine was to get a context and call methods:

func SomeHandler(w http.ResponseWriter, req *http.Request) {
    ctx := appengine.NewContext(req)    
    ctx.Infof("got request!")
}

Whereas the new way:

func SomeHandler(ctx context.Context, w http.ResponseWriter, req *http.Request) {
    log.Infof(ctx, "got request!")
}

The new way has one huge advantage, it allows you to pass any context which implements interface context.Context. It also requires to replace all appengine/log imports by a new package google.golang.org/appengine/log

The Go’ standard log library doesn’t have a concept of context at all.

Reasoning

I guess I should explain the advantage of using a context based log instead of standard one. In a case of task oriented application it allows the programmer to group all log messages related to given task. It’s a common practice to group messages by the place of the log function callback (the file and line where the log message is written). More sophisticated grouping includes process id and it may be a task id (or context in our case). Just imagine how much time it saves during debugging. It also allows much more sophisticated analysis of the server event flow without running in debugging mode.

I’ve created a tiny wrapper around both github.com/golang/glog library and google.golang.org/appengine/log: github.com/orian/utils/net/log. It should allow you to log both at App Engine and stand alone processes. Enjoy.

A real in depth explanation can be found in Go blog' article: Go Concurrency Patterns: Context

Tuesday, September 29, 2015

Upgrading an ASUS Z87 PLUS motherboard firmware

I needed to upgrade a firmware (in fact, it should have been done a long time ago). The steps goes as follow:

  1. Find a firmware: (https://www.asus.com/Motherboards/Z87PLUS/HelpDesk_Download/)
    http://dlcdnet.asus.com/pub/ASUS/mb/LGA1150/Z87-PLUS/Z87-PLUS-ASUS-2103.zip
  2. Unpack it and localize Z87-PLUS-ASUS-2103.CAP (or some other older version Z87-PLUS-ASUS-2004.CAP or Z87-PLUS-ASUS-1802.CAP)
  3. Prepare a FAT flash device
  4. Copy the firmware *.CAP file to flash root directory and rename to: Z87PL.CAP
  5. Plug flash into the 'flashback' USB port at the back of PC (it's next to ethernet with a green border around)
  6. Press flashback button on a motherboard, it's below a power plug, labeled as "BIOS_FLBK"
The FLBK_LED should start blinking (if it stops blinking means it cannot read firmware or the file is broken). It will take few minutes. DONE.


BTW. It sucks that Asus just doesn't say what should be the name of a firmware file. They could just use FIRMWARE.CAP and it would be fine. There are other ways of protecting people from themselves than encoding the model in a file name.

Tuesday, September 15, 2015

Slow compilation in Golang

Because I'm lazy and Go tends to not backport it's libraries, I've decided to migrate a project from go1.4 to go1.5. Everything would be great but the compilation process of the new version is noticeably slower than the old one.

Compile all your dependencies

What I've noticed some time ago is that if you change the version of Go the libs stay at old version. This causes them to be built on each go run you invoke. And if you use a library which automates rebuilding the binary on change like reflex () than you may wait long.

List all deps

To list all dependencies your application or library have you can invoke:
go list -f '{{.Deps}}' | tr "[" " " | tr "]" " " | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' > deps
(borrowed from from go-nuts)

Rebuild all packages

After that you can call:
go install -a $(tr -d '\n' < deps)

Update all packages

Alternative to the above, updates the code:
go install -u $(tr -d '\n' < deps)

Speedup

For my code it was from 15 seconds to below 1 second.

Apply to all subpackages in a project

As Craig Furman suggested in a comment (thanks):
go list -f '{{.Deps}}' ./... | tr "[" " " | tr "]" " " | \
  xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' | \
  xargs go install -a

Saturday, September 12, 2015

MySQL under the hood hurts

Recently I've been poking around MySQL, i've forgot how it's when someone makes a lot of decisions for you, e.g. that a column of type TIMESTAMP should have a default value set to CURRENT_TIMESTAMP or that on each row write it should be automatically updated to CURRENT_TIMESTAMP, and btw did you know, that NOW() and CURRENT_TIMESTAMP are not equal (check this)?

The simplest way to discover what MySQL have done for you is to either read the documentation (which is kind of long) or:

SHOW CREATE TABLE table;
This will print you the full command necessary to recreate the table in the same state as it's currently:
CREATE TABLE `table` (
  `user_id` int(11) DEFAULT NULL,
  `user_name` varchar(255) DEFAULT NULL,
  `tags` text,
  `created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `state` int(11) DEFAULT '0',
  `date_of_publication` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  `auth_fb_share` tinyint(1) DEFAULT NULL,
  CONSTRAINT `table_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`user_id`),
) ENGINE=InnoDB AUTO_INCREMENT=38 DEFAULT CHARSET=utf8

But why is it a problem?
Because by default CURRENT_TIMESTAMP inserts the value with timezone shift, but without the information about time zone. Let's say it's 2015-09-09 21:23 +02:00. The TIMESTAMP keeps the data as UTC (+00:00). The proper value should be: 2015-09-09 19:23.
But the CURRENT_TIMESTAMP will insert the data as: 2015-09-09 21:23 skipping time zone information.
This causes a shift of X hours depending what's a server timezone is.

Thursday, July 30, 2015

Go: a list of structure field names

package main

import (
 "fmt"
 "reflect"
)

type User struct {
 Name     string
 Age      int
 Password string
}

func GetFieldNames(s interface{}) []string {
 t := reflect.TypeOf(s)
 if t.Kind() == reflect.Ptr {
  t = t.Elem()
 }
 if kind := t.Kind(); kind != reflect.Struct {
  fmt.Printf("not a struct kind: %s", kind)
  return nil
 }
 var ret []string
 for i := 0; i < t.NumField(); i++ {
  field := t.Field(i)
  ret = append(ret, field.Name)
 }
 return ret
}

func main() {
 u := User{}
 fmt.Println(GetFieldNames(u))
 fmt.Println(GetFieldNames(&u))
}
At Go Playground: http://play.golang.org/p/6LDoCIKFeH
The function returns a slice of strings. Each is a name of a field. If a pointer is an argument the underlying type's field names are returned. The function ended up in: github.com/orian/utils/reflect. A documentation is browsable at: https://gowalker.org/github.com/orian/utils/reflect.
A short explanation:
  • First get a reflect.Type of a given argument s
  • If s' kind is a pointer, get a descriptor of underlying type.
  • If underlying type is not a struct, return error..
  • Iterate over fields, add field names and return the result slice.

Wednesday, July 1, 2015

TIL: clicked element reference in JavaScript console

I had a break from WebDev for around 2 years. I'm rediscovering things now. Today I've learnt: $0
  • Right click on the element of interest in your browser and select 'inspect element'. You should see the browser debugger with the element you clicked on highlighted.
  • The debugger allows you to access the currently selected element in the console as $0 variable.

Thursday, June 11, 2015

Monday, June 1, 2015

Endless extracting of tar.bz2 archive

Funny story. Over a year ago I've backed up a project directory to tar.bz2 archive. Today I've decided to check what's inside. I've encountered a frozen window of 'extract here' option from file context menu.

I've killed the extract process and rerun it from a terminal:
tar -xvvf archive.tar.bz2
After few normal lines i've seen smth (in fact thousands of such lines):
hrwxr-x--- user/group       0 2013-02-06 23:20 workspace/a/b/a/b/f.go link to workspace/a/b/f.go
hrwxr-x--- user/group       0 2013-02-06 23:20 workspace/a/b/a/b/a/b/f.go link to workspace/a/b/a/b/f.go
This clued me, that probably the archive kept the symbolic links. After Googling and reading tar manual I had this idea: let's remove a symlink from an archive:
tar --delete -f archive.tar workspace/a/b/a
This worked just well :-)

Thursday, May 28, 2015

pep8

I've got very comfortable with clang-format for C++ and gofmt for Go. Every time I have to use some other language i'm missing them very much. Today I've found a solution for Python. It makes life so much easier. The two awesome tools are:

pep8

pep8 is a tool to check your Python code against some of the style conventions in PEP 8.
https://pypi.python.org/pypi/pep8

autopep8

autopep8 automatically formats Python code to conform to the PEP 8 style guide. It uses the pep8 utility to determine what parts of the code needs to be formatted. autopep8 is capable of fixing most of the formatting issues that can be reported by pep8.
https://github.com/hhatto/autopep8

Wednesday, April 29, 2015

Torch7 weird error and Google Search without a solution

The below is an error one gets when working with Torch7 from time to time:
/usr/local/bin/luajit: /usr/local/share/lua/5.1/nn/Tanh.lua:4: attempt to index field 'nn' (a nil value)
stack traceback:
 /usr/local/share/lua/5.1/nn/Tanh.lua:4: in function 'updateOutput'
 /usr/local/share/lua/5.1/nn/Sequential.lua:29: in function 'updateOutput'
 /usr/local/share/lua/5.1/nn/Sequential.lua:29: in function 'forward'
 main.lua:93: in function 'opfunc'
 /usr/local/share/lua/5.1/optim/sgd.lua:43: in function 'sgd'
 main.lua:117: in function 'train3'
 main.lua:142: in main chunk
 [C]: in function 'dofile'
 /usr/local/lib/luarocks/rocks/trepl/scm-1/bin/th:131: in main chunk
 [C]: at 0x00406260
Reading a bit about similar errors(1) I've concluded that probably the type of Tensor is wrong. I work on torch.CudaTensor objects, why the problem? I've forgot to import the correct library:
require("cunn")

Wednesday, April 22, 2015

Reflex - trigger an execution of command on a file change event

How many times you were working on some tiny project that needs to be rebuild / restarted after you modify the code? I guess often.

After experimenting with some solutions I found an ultimate one: https://github.com/cespare/reflex
Installation:
go get github.com/cespare/reflex
Automatically recompiling markdown file into html every time the file is changed:
reflex -r '\.md$' blackfriday-tool -page article.md article.html
Or you can restart Go app when saved files:
reflex -r '\.go$' -- go run main.go

Wednesday, April 15, 2015

xmonad and screen off

In Ubuntu with default windows manager it just works, with Xmonad it doesn't, turning the screen off and making the eco-people happy (saving your electricity bill).

The inspiration for solution comes from:
http://superuser.com/questions/590225/disable-screen-turn-off

Checking xset tool in terminal shows smth:
xset
usage:  xset [-display host:dpy] option ...
    To turn bell off:
 -b                b off               b 0
    To set bell volume, pitch and duration:
  b [vol [pitch [dur]]]          b on
    To disable bug compatibility mode:
 -bc
    To enable bug compatibility mode:
 bc
    To turn keyclick off:
 -c                c off               c 0
    To set keyclick volume:
  c [0-100]        c on
    To control Energy Star (DPMS) features:
 -dpms      Energy Star features off
 +dpms      Energy Star features on
  dpms [standby [suspend [off]]]     
       force standby 
       force suspend 
       force off 
       force on 
       (also implicitly enables DPMS features) 
       a timeout value of zero disables the mode 
    To set the font path:
  fp= path[,path...]
    To restore the default font path:
  fp default
    To have the server reread font databases:
  fp rehash
    To remove elements from font path:
 -fp path[,path...]  fp- path[,path...]
    To prepend or append elements to font path:
 +fp path[,path...]  fp+ path[,path...]
    To set LED states off or on:
 -led [1-32]         led off
  led [1-32]         led on
 -led named 'name'   led off
  led named 'name'   led on
    To set mouse acceleration and threshold:
  m [acc_mult[/acc_div] [thr]]    m default
    To set pixel colors:
  p pixel_value color_name
    To turn auto-repeat off or on:
 -r [keycode]        r off
  r [keycode]        r on
  r rate [delay [rate]]
    For screen-saver control:
  s [timeout [cycle]]  s default    s on
  s blank              s noblank    s off
  s expose             s noexpose
  s activate           s reset
    For status information:  q
    To print version: -version

The interesting part is dpms. To force turn off the screen one has to run:
xset dpms force off
And it would be great to integrate it with a 'lock screen' keyboard shortcut in Xmonad configuration:
    -- lock screen
     , ((modm .|. shiftMask, xK_l     ), spawn "gnome-screensaver-command -l && xset dpms force off")