Hi guys,

This is something I encounter on daily basis, so, I decided to write a little bit about how I solve this problem in javascript.

Let’s say I have Array of objects and want to normalize it for inserting into database.

Some object might miss some data, some arguments might need to be processed, etc. and this code can get really messy sometimes.

What our goal here is to try to avoid putting column list 5 times and try to do it with as less code as possible.

I tried a lot of different ways to do this and this is the one I like the most.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

let people = [
	{
		name: "Petar Petrovic"
	},
	{
		firstName: "Petar",
		lastName: "Petrovic"
	},
	{
		name: "Petar Petrovic",
		hasMedicalProblems: true
	}
];
console.log(JSON.stringify(people, null, 4));
let head = {
	"name": function(person){
		return person.name || (person.firstName + " " + person.lastName) || "Unknown";
	},
	"hasMedicalProblems": function(person){
		return person.hasMedicalProblems || false;
	}
}
people = people.map(function(person){
	let newObject = {};
	Object.keys(head).forEach(function(key){
		newObject[key] = head[key](person);
	});
	return newObject;
});
console.log(JSON.stringify(people, null, 4));

Comment and share

If you ever had to do portforwarding on network that was not yours, you know it can be pain in the ass sometimes and you most probbably do not want to deal with it. o

This method is very easy to do and will allow you to expose your port to public and even stop local computers from sniffing data to and from that port.

How is this done in theory

Well, if you ever used SSH, you probbaly know it is just encrypted tunnel…

What you maybe did not know is that it can carry a lot more than text. (you cna use it to copy files, to open graphical application, as a SOCKS proxy, etc.)

One of things a lot of people do not know about is that you can also use SSH to forward ports from one side of tunnel to another…

Lets say your computer can access some port and someone in network you are connecting to can not…

What you can do is specify port forwarding in SSH configuration.

Let’s say you want to forward local port to WAN…

What happens is remote machine you are connected to will listen on specified port and when someone connect to that port, your computer will connect to local port and data will be tunneled inside SSH.

How to do this in practice

This is actually very simple

1

ssh -R 221:localhost:22 server

This will make server listen on port 221 and when someone connects to it, you will connect to localhost:22 and connection will be established.

One small trick is that in OpenSSH (implementation of SSH cost often used) server will listen only on loopback interface (localhost) and not on other interfaces.

To solve this, we need to add:

1

GatewayPorts yes

to the end of /etc/ssh/sshd_config and restart SSH server. (systemctl restart sshd for SystemD or service ssh restart)

Comment and share

phpfbsession

Script for maintaining session from browser for facebook automation

Adding session

First, you need to get cookies from your web browser.

To do that, I would recommend cookies.txt

After you got your cookies, you should run:

1

php addAccount.php

Now, paste cookies and pres Enter twice!

Keeping session in working order?

To keep session in working order make sure you do not log out in browser or simply delete cookies in browser.

How to get something from facebook?

1
2
3
4
5
6
7
8
9
10
11

<?php
require_once("./vendor/autoload.php");
require_once("./Config.php");
require_once("./Parser.php");
$config = new Config;
$parser = new Parser($config->get("accounts"));
echo $parser->get("https://www.facebook.com/");

CLI menu

If you prefer to run CLI menu, you can do so by:

1

php index.php

Troubleshooting

I have problem with session, what do I do?

1

php checkAccounts.php

If problem stays, just delete that account from config.json and readd it.

Authors

• nemanjan00

Comment and share

Hi guys,

I created a few applications that stream torrents, so, I get asked quite a bit about it.

Today I am going to write a little bit about how it works and how to utilize it.

Where do I get torrents from?

First of all, to get torrents, you can use search engine/file someone sent you/create your own torrent…

So, what torrent file/megnet link is?

Torrent file/magnet link

When you get torrent in any form, what you actually get is hash of that torrent.

It can be either looked up on Torrent trackers (web services) or DHT (distributed hash table).

The part that is actually looked up is hash of that torrent.

That hash is used to identify and verify that data integrity.

What do we do with data from tracker/DHT?

What we get from DHT is list of peers that have torrent we are looking for.

What happens next is we download data about torrent from those peers. (File list, sizes, etc.)

Ok, how do we download files now?

What is happening now is we are reaching out to peers and they are offering parts they have.

For streaming, we are simply accepting parts that are near area we need and deny those that we do not need at the momment.

How can we implement it?

If you are writing your application in Node.JS, what I would recommend to you is torrent-stream package. It has very simple API and it is very easy to pass it through HTTP layer.

If you are writing your application in any other language, I would recommend using peerflix which is CLI version of torrent-stream.

Comment and share

Hi guys, what I had to do few days ago is dynamically change speed of audio playback.

Terminology

Sample is sound amplitude.

Audio file is just encoded/compressed list of samples.

Samplerate is number of samples per second. (the one I will be using is 44100)

Buffer is Array of samples that are going to go out of your speaker. (Common sizes are 256, 512, 1024, etc. sample rates)

What did I end up using?

Technology I was using for that is Web Audio API.

AudioNode

AudioNode is basicaly Object which you pipe audio into and/or get audio out of it.

AudioNodes connect to each other just like any audio equipment would in real life.

For example you maybe have microphone, speaker, spectrum analyzer with passthrough and amplifier.

You can connect for example microphone -> amplifier -> spectrum analyzer -> speaker

Getting audio inside system

What most of the people in Web Audio API would use if they wanted to play music inside it is MediaElementAudioSourceNode (implementation of AudioNode that gets sound from HTML5 audio tag).

It simply plays audio and that is it.

Problem with this one is that it is impossible to change speed. (might be possible to slow it down, but I did not find any ways to make it play music faster)

decodeAudioData

What I ended up doing is getting mp3 using XHR as ArrayBuffer and running decodeAudioData from audio context on it.

That gave me back buffer containing whole audio:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

module.exports = function(url){
	return new Promise((resolve, reject) => {
		const audioCtx = new (window.AudioContext || window.webkitAudioContext)(); // Getting audio context. Anything you do with Web Audio API requires use of audio context
		const xhr = new XMLHttpRequest();
		xhr.open('GET', url, true);
		xhr.responseType = 'arraybuffer'; // This will make sure I get response in format I need for passing it to Web Audio API
		xhr.onload = function(e) {
			var audioData = xhr.response;
			audioCtx.decodeAudioData(audioData, function(buffer) { // This will decode that audio file and return buffer using callback. 
				resolve(buffer);
			}, function(e){
				reject(e);
			});
		};
		xhr.send();
	});
}

What I can do now is take data from buffer at any rate.

Scaling audio using input audio buffer

Now that I have list of amplitudes I can play that audio back.

Web Audio API does not have any way to generate sound, just process it (at least I did not find anything like that).

What I ended up doing is using ScriptProcessor (AudioNode for transforming sound).

I did not hook up any audio sources to it, just hooked it up to sink.

transformationFunction is function that is getting called each time you need to process audio. Inside it I am taking audio from music buffer and putting it into output buffer.

Calculating sample in output buffer

What I have is huge buffer that contains song and I have very small buffer I am writing to.

I have counter i which tells me where I am inside input buffer.

What I do is iterate inside output buffer and calculate what I want to take from input buffer for that sample.

Let’s say playback speed is 1.5.

What I need to put into output buffer are samples 1.5, 3, 4.5, etc.

But, there are samples 1 and 2, and there is no sample 1.5, what do I do now?

Well, what you can do is:

1

sample = 0.5 * input[0] + 0.5 * input[1];

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

module.exports = function(audioCtx, buffer, outputNode){
	let i = 0;
	let cnt = 0;
	let speed = {
		speed: 1,
		playing: 1,
		buffer: undefined,
		transformationFunction: function(audioProcessingEvent) {
			let outputBuffer = audioProcessingEvent.outputBuffer;
			let buffer = speed.buffer;
			let oldI = i;
			let end = false;
			for(chan = 0; chan < 2; chan++){
				i = oldI;
				let position = 0;
				let input = buffer.getChannelData(chan);
				let out = outputBuffer.getChannelData(chan);
				do {
					i += speed.speed * speed.playing;
					if(i < 0 || i >= input.length){
						end = true;
						speed.playing = false;
					}
					if(!end){
						let floor = Math.floor(i);
						let diff = i - floor;
						out[position] = input[floor] * diff + input[floor + 1] * (1 - diff);
					} else {
						out[position] = 0;
					}
					
					position++;
				} while(position < out.length);
			}
		},
		init: function(audioCtx, buffer, outputNode){
			const scriptNode = audioCtx.createScriptProcessor(512, 0, 2);
			speed.buffer = buffer;
			scriptNode.onaudioprocess = speed.transformationFunction;
			scriptNode.connect(outputNode);
			return speed;
		}
	}
	return speed.init(audioCtx, buffer, outputNode);
}

Comment and share

Dokku is extremly simple PaaS (platform as a service) .
Philosophy of this software is quite simple. It is as simple as possible clone of Heroku.

Advantage of system like this is that you do not have to (but you can) think about container application is going to run inside but only about application itself. All you need to do is git push and Dokku will make fully functional container out of it and deploy it.

What happends when I do git push?

First thing you need to know about Dokku, Heroku and similar software is that they use something called buildpacks. Buildpack is collection of additional software required to build and run your application. For example there is PHP buildpack that contains PHP, ngginx, apache, composer, etc.

What is buildpack and how do we use it?

Buildpack is collection of 3 scripts with specific role.

• bin/detect - Role of this script is to detect if that buildpack was developed to be used with application like yours. This one is skiped if user already defined buildpack and is used for autodetect of language only.

• bin/compile - Role of this script is to prepare your application for running (installing dependencies, building, minification or stuff like that)

• bin/release - Role of this script is to tell which commands should you run to start this container.

As you have probbably already figured out, first we need to figure out which buildpack should be used for your application.

It can be either specified by user or detected by Dokku/Heroku/etc. by running all buildpacks against script until detect script returns true.

After that, compile script is run to prepare your application for deployment,

After that, for Dokku to be able to know how to run container, release return list of commands.

Example:

1

web: node index.js

If your application does not have Procfile, the one from release script will be used.

There is just one step left after this… Image is commited and ran.

The only remaining whing is communication between your application and outside world. For anyone to be able to connect to your application from outside, your application needs to listen on specific port.
For default http port, dokku is setting global variable called PORT to port number. (And nginx is used to proxy to your application)

You can also configure Dokku to forward other ports to your application.

What are advantages of system like this?

There are a lot advantages but thrse are the ones that got me to switch:

• Consistent enviroment - You are sure your application will act the same way on any server that uses Dokku.

• Easy deployment and fast first time configuration

• There is big community and a lot of plugins for it.

Comment and share

My friend was quite confused about OAuth 2 today (facebook specifically) so I decided to write a couple of tricks for facebook and instagram authentication.

Types of authentication on Instagram/Facebook

There are two types of authentification.

• With backend callback
• With frontend callback (facebook specific implementation)

Backend callback

On your redirect url you are getting code GET argument and that code is what you use to get token.

Frontend callback (facebook implementation)

When you trigger authentication from javascript, facebook sdk opens popup for authentication and when popup authentification is finished, window will get closed and you will get token back.

It is all OAuth 2, why is facebook different?

Most modern applications today are modern and it is just not natural experience to have redirect and to reaload page.

A lot of people would like to use same style of authentification for Instagram as for facebook.

What is the actual difference between facebook auth and normal OAuth2?

What facebook did is build one more layer on top of OAuth.

They developed backend that just returns token to your frontend application.

How do I do same thing for instagram?

Before we implement the same thing for instagram, we need to get familiar with couple of concepts.

Opening window

1

var window = window.open(url, windowName, [windowFeatures]);

When you open popup window, you are getting window object back and now, you can reference it.

In window you opened you can reference parent using opener

Cross window communication

Sending data

1

window.postMessage(message, targetOrigin, [transfer]);

In this case, you are sending message which is data object to window if target origin matches with origin of that window.

Transfer is optional object that will be removed from your scope and added to that message. It can also contain stuff that is not data.

Listening for data

1
2
3
4
5
6
7
8
9

window.addEventListener("message", receiveMessage, false);
function receiveMessage(event)
{
  if (event.origin !== "http://example.org:8080")
    return;
  // ...
}

Implementing instragram login

What we now need is backend script that will grab token from code and send it to window that called it.

1

opener.postMessage("put token here", "origin"); // To make sure noone steals token, use origin

In window that needs to login, we are opening instagram login in popup and put link to that backend script in redirect.

1

window.open("https://api.instagram.com/oauth/authorize/?client_id=APPLICATION_ID&redirect_uri="+window.location + "auth/instagram"+"&response_type=code");

Now, we need to wait for token response

1
2
3
4
5
6

window.addEventListener("message", receiveMessage, false);
function receiveMessage(event){
	$window.removeEventListener("message", receiveMessage, false);
	// Token is in event.data
}

Comment and share

So, I just had a problem starting Atom editor.

I do not run it too often (mostly for search of projects).

For some reason, it did not want to start.

So, I wanted to debbug it.

How to debbug Atom?

When I start it, it just goes to background…

After a little of investigation (cat $(which atom)) I figured out output was piped into ~/.atom/nohup.out…

What was the problem?

There was error in output and it said that module inside atom was built with different version of Node.

From my experience, with packaged applications, that most often means it was ran by wrong version of electron.

I also have electron installed globally and that version is default.

Fix?

Well, I now need to make it run with electron installed by package manager…

First I listed files of that package:

1

pacman -Ql electron | grep bin

I got /usr/bin/electron as a response.

Now, I found command that starts atom in cat $(which atom) and just replaced electron with full path of one from package manager.

1

/usr/bin/electro --app=/usr/lib/atom

It worked as expected so, now I just created alias for atom…

1

alias atom="/usr/bin/electron  --app=/usr/lib/atom"

And that was it.

Comment and share

If you are anything like me, it happens to you also that you battery just runs out and laptop turns off.

The problem comes if you were just doing kernel update…

That is the problem I had last night…

So, how did I fix it?

Getting shell on that computer.

I grabbed first linux installation I found laying around.

It was Ubuntu Server…

It does not have live system, but, when you start installation, you can just switch to different tty (Ctrl+Alt+F2)..

Getting shell inside that system.

Now, what you need to do is chroot into your installation of arch.

Finding your partitions

All od your partitions should be enumerated in /dev/sd* or in /dev/mapper/* (for lvm).

If you use have encrypted lvm/partition, you need to first run

1

cryptsetup luksOpen /dev/sdX2 name

And, if I remmember well, that partition/lvm will appear in:

1

/dev/mapper/name

Mounting partitions

Now that you found/decrypted/etc. ypur partition, we need to mount them.

We are going to take /mnt as base and mount your system inside.

For example:

1
2
3

mount /dev/sda2 /mnt
mount /dev/sda1 /mnt/boot
mount /dev/sdb1 /mnt/home

Chrooting

If you are using arch live, you probbably have arch-chroot command.

What you should do is just:

1

arch-chroot /mnt

If you are not using arch live, you first need do mount /sys, /dev and /proc

1
2
3
4
5
6
7

cd /mnt
mount --rbind /sys sys/
mount --rbind /dev dev/
mount --rbind /proc proc/
chroot .

Ok, I have shell, what now?

Well, now you just need to install any version of kernel form cache…

1

pacman -U /var/cache/pacman/pkg/linux- # press tab and pick any version

Now, just reboot computer and update system. That should be it.

Comment and share

party-youtube-downloader

Run it localy

1
2

sudo npm install party-youtube-downloader -g
party

Deploy to Heroku

Written in

Backend

• Node.JS
  • Express
  • ytdl-core

Frontend

• Bootstrap
• AngularJS
  • Angular-Bootstrap
  • Angular-Animate

Development Enviroment

• Bower
• Grunt

How to develop?

1

grunt init; grunt watch

and in another terminal

1

nodemon ./index.js

Authors

Nemanja Nedeljković

Comment and share