Optimize MySQL / MariaDB for Import


Tried to import 340MB SQL file into MariaDB via its console, it took hours to get finished on XAMPP 8.1.6 in Windows 10 with 16GB RAM.

Then, I stop MariaDB, open my.ini file and set these options

innodb_buffer_pool_size = 4G
innodb_log_buffer_size = 256M
innodb_log_file_size = 1G
innodb_write_io_threads = 16
innodb_flush_log_at_trx_commit = 0
view raw my.ini hosted with ❤ by GitHub

Then, I start again MariaDB, import on different database

mysql -u root -p
SET FOREIGN_KEY_CHECKS=0;
use database_name;
SOURCE data.sql;
SET FOREIGN_KEY_CHECKS=1;
exit;

Now it took less than 5 minutes to get finished 🙂

* Image by Benjamin Lehman and Ray Hennessy

PyTesseract: Extract text from certain image region


I already have image region coordinate, I want to crop that region (for backup) and feed the cropped image to PyTesseract then extract the text from that image.


You can use OpenCV selectROI to manually select certain image region, beware of its coordinate notation, selectROI use x1,y1,width, height notation, while PIL use x1,y1,x2,y2 notation.

import cv2
image = cv2.imread("image.png")
x = cv2.selectROI("Select area", image, fromCenter=False)
print("Selected box: ", format(x))
view raw selectROI.py hosted with ❤ by GitHub

In this example I just use PIL, here’s the code…

from PIL import Image
from pytesseract import pytesseract
path_to_tesseract = r'C:\Program Files\Tesseract-OCR\tesseract.exe'
pytesseract.tesseract_cmd = path_to_tesseract
img = Image.open("original_image.png")
# img.show()
box = (297, 10, 584, 50)
img2 = img.crop(box)
croppedImage = "cropped_image.png"
img2.save(croppedImage)
# img2.show()
img3 = Image.open(croppedImage)
text = pytesseract.image_to_string(img3)
print(text)

* Image by David Pupaza

Install Kong on Ubuntu


Kong is a pretty good API Gateway, can be installed in many ways. Installing via apt will make it easier to update

$ sudo apt-get install -y apt-transport-https curl lsb-core
$ echo "deb https://kong.bintray.com/kong-deb `lsb_release -sc` main" | sudo tee -a /etc/apt/sources.list
$ curl -o bintray.key https://bintray.com/user/downloadSubjectPublicKey?username=bintray
$ sudo apt-key add bintray.key
$ sudo apt-get update
$ sudo apt-get install -y kong

Flask in Windows


I’m using Python 2.7.17 on Windows 8, virtualenv already installed (pip install virtualenv), here’s my simple step to create hello world style app

$ set FLASK_APP=app.py
$ set FLASK_ENV=development
$ virtualenv venv
$ cd venv/Scripts
$ activate
$ pip install flask
$ cd ..
$ touch app.py
$ subl app.py
-------------------------------
from flask import Flask

app = Flask(__name__)

@app.route('/')
def welcome():
return '<p>welcome to the jungle</p>'

if __name__ == '__main__':
app.run(debug=True)
-------------------------------
$ python app.py

or

$ flask run
Then, point browser to http://localhost:5000

 

Codeception – Create Helper for Reusing Token


create the file on tests/_support/Helper/ folder, to reuse the token you need to create hook _before

 

tests/ApiCest.php
-----------------
<?php 

class ApiCest 
{
    private $token;
    
    
    function createMeetingTest(ApiTester $I) {
        $data = array(
            "meeting_title" => "Meeting title",
            "meeting_date" => "2019-11-21 10:00:00"
        );
        
        $this->token = $I->getToken();
        
        $I->wantToTest('Create a meeting');
    	$I->amBearerAuthenticated($this->token);
		$I->sendPOST('/meetings', $data);
		$I->seeResponseCodeIs(200);
		$I->seeResponseIsJson();        
    }

    
}

tests/_support/Helper/Api.php
-----------------------------
<?php
namespace Helper;

class Api extends \Codeception\Module
{
	public $credentials = array(
		'username' => 'remo.harsono',
		'password' => '12345678'
	);
    
	private $jwt;


	public function getToken() {
		return $this->jwt;
	}

    public function _before(\Codeception\TestInterface $test) {
		$I = $this->getModule('REST');
		$I->haveHttpHeader('Content-Type', 'application/json');
		
		$I->sendPOST("/auth/login", $this->arrCredentials);

		$I->seeResponseCodeIs(200);
		$I->seeResponseIsJson();

		$response = $I->grabResponse();
		$json = json_decode((string)$jesponse);
		
		if ($json->result == 'OK') {
			$this->jwt = $json->jwt;
		} else {
			// ....
		}
	}
}


sample json output when calling auth/login
------------------------------------------
{
  "result": "OK",
  "jwt": "blablablabla",
}

Install VSFTPd on AWS EC2



sudo apt install vsftpd
sudo cp /etc/vsftpd.conf /etc/vsftpd.conf.original
sudo nano /etc/vsftpd.conf

anonymous_enable=NO
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd.chroot_list
ls_recurse_enable=YES
local_root=/var/www
seccomp_sandbox=no
pam_service_name=ftp
pasv_enable=YES
pasv_min_port=13000
pasv_max_port=13100
port_enable=YES
pasv_address=__your_ec2_domain__
pasv_addr_resolve=YES

sudo useradd myftp
sudo passwd myftp
sudo usermod --home /var/www/ myftp

sudo addgroup ftpusers
sudo usermod -a -G ftpusers myftp
sudo usermod -a -G www-data myftp
sudo usermod -g ftpusers myftp

sudo nano /etc/vsftpd.chroot_list
ubuntu

sudo nano /etc/ssh/sshd_config

Match Group ftpusers
ForceCommand internal-sftp
ChrootDirectory /var/www/
PermitTunnel no
AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no

sudo service ssh restart
sudo systemctl restart vsftpd

Set you EC2 Security Group to allow incoming tcp traffic to port 20-21, 13000-13100
Connect from you ftp client and set connection to passive mode

 

Nodejs – Error: ENOENT: no such file or directory, scandir ‘node_modules/node-sass/vendor’


I’ve experience weird moment today when trying to run my friend’s gulp script which call node-sass, it throw this error

Error: ENOENT: no such file or directory, scandir '**/node_modules/node-sass/vendor'

I call it weird because normally it has ‘vendor’ folder.

After Googling, visiting some stackoverflow pages, the solution is simple: just invoke these two commands

nodejs node_modules/node-sass/scripts/install.js
npm rebuild node-sass

then you’re good to go 🙂