Alternative for Samba or ftp to Real Time Sync for Your Project Folder to Server - Push Files to Server on Every Change
on Blog
Some time we have to develop software on own device but also we have to compile or run at remote computer.
Why I am using remote compile or debug ?
First reason is test publicly and access at everywhere. Second reason is some times I am developing on my phone. Compiling on phone is not good idea.
Third reason is my developing and compile environment is different. My computer is not well support on Linux system, so also most of the time my projects is about on Linux. Due to this requirement, I have to use Linux on virtual machine and I am editing files on network.
Mirrorfy is avaible. Visit project.
Lets consider default solutions.
I am using samba for realtime debugging in my projects on local systems. But samba does not meet my needs. My IDE is atom and while working on remote project atom is crashed sometimes and I have to restart my IDE. Deleting on some file on samba with Atom causes an error and files is not deleted, changing .gitignore file also is not possible due to permission error. Third problem is when the remote is really far, samba protocol or others is very slow down and this affects IDE or freeze the ide. Fourth reason is samba has a lot of vulnerabilities.
Anyway these reasons enough for me to find a new method.
What is my option ?
Few weeks ago, I transfer my server to other company. I am used rsync for transfer my docker containers. You can access at here
The transporting speed is impressed to me and my new plan is use rsync to synchronize to my files.
Working Logic
Firstly I have to say this is only one way sync solution because in this project we only use bash for the scripting. I also planned and designed a two-way and multi-user solution, but I don’t not have time to write it down software, due to many projects and designing new systems.
In the client side we have file and folder watcher application. When the changes happened the application is trigger while loop and bash will be execute the rsync to upload files to server.
Server side is only handling the requests which is default by ssh.
Client Side Configuration
Installing rsync and inotifywait
The system requires only rsync, inotifywait, screen and ssh application. SSH and bash is already installed by default, so we can install rsync and inotifywait on the system.
Note: Do not forget, run your package installer in root. if you are not on root user use sudo command to run package manager with admin privileges.
# Ubuntu, Debian
apt install rsync inotify-tools screen
# CentOS, Fedora, RedHat
yum install rsync inotify-tools screen
# Alpine
apk add rsync inotify-tools screen
Generate ssh-key
To the authorization we will use ssh keys. Before creating ssh key check your ssh public key. If it’s available you don’t need to create ssh-keys.
echo ; cat ~/.ssh/id_rsa.pub && echo || ssh-keygen
# If the key available output like this. This is your key
ssh-rsa AAAAB3NzaCfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdgsadgasdgasdgasdgFDrKtgOkVhmevubPrezH0ha8b8fk9pqQSObCHj6dybHMbt root@sdfgsdg
# If the key is not available output will be like this
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:XOVt8uasdfwhHrNITlMFX213fghuilohgDexTStuIihPyI root@sdfgsdg
The Sync Script
Bash script will be follow your project folder. If something is changed the script will be sync your folder. In my case I use this script on windows 10. My project path is /mnt/c/Users/ahmet/go/src/github.com/AhmetOZER/dibow on my laptop. In server side path is /my/projects/golang/. So my project which is dibow will be on server side in /my/projects/golang/dibow path.
Run nano sync1.sh command add codes under below, change your directory based on your project path and also our server side path, server ip, port.
#!/bin/bash
while inotifywait -r -e modify,create,delete,move /mnt/c/Users/ahmet/go/src/github.com/AhmetOZER/dibow
do
rsync -avzhp --delete -e "ssh -p 22" /mnt/c/Users/ahmet/go/src/github.com/AhmetOZER/dibow syncuser@myserverip:/my/projects/golang/
done
Run chmod +x sync1.sh to get executable permission to sync.sh file.
Adding to startup
For Linux
Visit StackExchange to how to start applications automatically on login. Your command is bash -c "screen -S sync1 -dm bash ~/sync1.sh" this.
For Windows
Run cmd as a adminstrator
# To add startup
schtasks /Create /TR "bash -c 'screen -S sync1 -dm bash ~/sync1.sh'" /RU %USERNAME% /TN sync1 /SC ONLOGON /IT
# To run now
SCHTASKS.EXE /RUN /TN "sync1"
Server Side Configuration
Installing rsync
The system requires only rsync and ssh application. Again, SSH is already installed by default, so we only install rsync on the system.
# Ubuntu, Debian
apt install rsync
# CentOS, Fedora, RedHat
yum install rsync
# Alpine
apk add rsync
Create new user for the system security.
Using root is not good idea for automated solutions. Due to we will create new user for the protecting system and disable password login.
adduser syncuser --disabled-password
# Example output will be like this.
Adding user 'syncuser' ...
Adding new group 'syncuser' (1000) ...
Adding new user 'syncuser' (1000) with group 'syncuser' ...
Creating home directory '/home/syncuser' ...
Copying files from '/etc/skel' ...
Changing the user information for syncuser
Enter the new value, or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] y
Note: If you want to sync your web folder realtime, you have to add new user to www-data group to prevent permission error.
usermod -a -G www-data syncuser
Adding client ssh keys to new account.
In client side run cat ~/.ssh/id_rsa.pub command and copy the output. Add output to your authorized_keys file on syncuser to your server.
su syncuser
cd
ssh-keygen # To create .ssh folder and apply required permissions
cat <<EOF>> .ssh/authorized_keys
PASTE your client key to here
ssh-rsa AAAAB3NzaCfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdgsadgasdgasdgasdgFDrKtgOkVhmevubPrezH0ha8b8fk9pqQSObCHj6dybHMbt root@sdfgsdg
EOF