In one of previous posts I said that pairwise layer doesn't seem to work...

Well, I was wrong: after checking on higgs-boson dataset from kaggle I found out that this kind of neural network works much better than traditional ones! Hurrah!

Though, much worse then GBDT, but after building AdaBoost over neural network I was able to get comparable (or just the same) quality. The only problem is GBDT trained in minutes, while it took ~24 hours for boosting over NN to train.