Friday, November 1, 2024

Docker: Setting up Gitea with Postgres using TLS via Step-Ca

 

Docker: Setting up Gitea with Postgres using TLS via Step-Ca

Project Github Link


I started with a simple objective to learn git as it was becoming more essential in my work in infrastructure. I was coming across more CI/CD pipelines with git repos and also thought it would be good to manage my own scripting in git. I also wanted to combine this with docker to improve my knowledge of docker and start using it for my setup on my homelab. 


It got complicated via some cyber security reasons around storing passwords in clear text. Some security frameworks state that this issue is an instant fail/breach. And my chosen Gitea implementation of git stores the database password in clear text. In the end it also stores tokens in clear text so I abandoned thoughts of bringing Gitea into my work setting but still using it on my homelab setup. I did mask the Gitea password by using Postgres pure “cert” authentication and can use a “-” character in the Gitea ini file. I moved the tokens out into dedicated files separate from the ini file and inside the container (not the docker volume). 


Also complicated by initially using one combined docker compose implementation and then wanting to use postgres in another use case. So I chose to split it all out into individual compose and docker files (that are dependent) and combine them into a controlling compose file using the “include” notation. It does not use any provided docker hub or other setup, I chose to install the software during the build on a prepared base image. The version control is in the .env file. And also the entry command scripts are based on one general control script “template_start_script.sh” to try and keep consistency in the approach. 


The following is based on downloading the git repo and setting up the pre-reqs, it's been tested on Alma and Ubuntu linux systems.


*All steps need to be executed by the root user unless otherwise stated

Step 1 setup local user and directory


Creating a dedicated docker user in this example dockeruser1 - if this is changed the .env file needs to be adapted in step 2


The directory where the downloaded project is stored can be adapted as well.

The example uses  /srv/docker-config

Example using the name of the downloaded zip as gitea-postgres-stepca-docker


# su to root  e.g. 

sudo su - 

useradd -m -s /bin/bash -u 1245 dockeruser1

mkdir /srv/docker-config

     

cd /srv/docker-config


# Install zip if required - (“apt install -y zip” on ubuntu) or (“dnf install -y zip” alma/rocky)

unzip {downloaded zip file} -d .

Run the initial script to copy the TEMPLATE env and info files into the actual files used in the setup



cd /srv/docker-config/gitea-postgres-stepca-docker

./initialGitHubScript.sh




Edit the .env file to adapt the entries in bold to the requirements on the host

The example would be the following,

Step 2  Adapt the .env file


Change bold entries if required and ignore any other settings in the .env file

Local docker user and group needs to match the user created in step 1


# check Env values

#LOCAL_SETUP - used by first_setup.sh scripts

LOCAL_DOCKER_USER=dockeruser1

LOCAL_DOCKER_GROUP=dockeruser1

LOCAL_DOCKER_VOLUME_DIR=/docker-vol1

DOMAIN=rjruss.org

MX_SHARED_ENV_GROUP=5501

BASE_HOST=newhost-

STEP_HOST=${BASE_HOST}step

POSTDB_HOST=${BASE_HOST}post

GITEA_HOST=${BASE_HOST}gitea

#  STEP value for default app is the container name - indicated by *-run host

MX_APP_STEP_PORT=9011

MX_APP_GITEA_PORT=3011

MX_APP_GITEA_SSH_PORT=2222



The above would be allow the services to be accessed 

  1. Via the docker user

Adapt the dockeruser1 - group dockeruser1 is used by default

  1. Via the shared group 

Adapt the group id of 5501 a new group will be created with that gid

  1. Via a new directory/volume (it will be created if it does not exist)

Adapt the /docker-vol1

  1. Via the following domain, hostname and ports

rjruss.org = this needs to be adapted to the required domain

newhost-step = for step-ca hostname

newhost-post = postgres hostname

newhost-gitea = gitea hostname


Step-ca port = 9011

Gitea port = 3011

Gitea ssh port = 2222


(postgres is not by default open to connections other than on the docker network and uses the standard postgres port 5432 in the docker network)

Step 3 adapt the age related password files


The age (see step 4) command is used to store passwords in an encrypted file on a docker volume. These can be shared between containers via the bash scripts in the base docker container build.


These .*.info files should be deleted ***after completing the step 4 process***

#Gitea DB connection password (not used but can be used if postgres config adapted)


#Edit file .GITEA.info

DB_PW="testing!1234!"


#Edit file .POST.info

POSTGRES_PASSWORD="testing!123"


#Edit file .STEP.info 

PW="testing1234"





Step 4 first setup script


The first_setup.sh script performs the following and the setup will fail if these are not installed.


  • Install standard operating system packages for docker, wget, curl, jq, ca-certificates, git and openssl

Also downloads and installs the following

  • age age is a simple, modern and secure file encryption tool, format, and Go library.” 

https://github.com/FiloSottile/age

  • yq a lightweight and portable command-line YAML, JSON and XML processor”

https://github.com/mikefarah/yq


The script also runs the scripts to create docker volumes based on the directory defined in variable LOCAL_DOCKER_VOLUME_DIR above. Also it creates a dedicated docker network for the containers in this project. It also uses the age command to encrypt passwords/secrets in the related docker volumes.


sudo su - 

cd /srv/docker-config/gitea-postgres-stepca-docker

./first_setup.sh



After running the script the following volumes and network will be available.


docker volume ls

DRIVER    VOLUME NAME

local     mx_base_gitea_vol1

local     mx_base_info_vol1

local     mx_base_keys_vol1

local     mx_base_pgdata_vol1

local     mx_base_step_vol1


docker network ls

NETWORK ID     NAME           DRIVER    SCOPE

f9137c47b08f   bridge         bridge    local

eb0a2e835235   host           host      local

e9d5110fe553   mx-base-net1   bridge    local

854f22748de4   none           null      local


**** delete these password files  ****ENSURE YOU KNOW THE PASSWORDS before deleting**** the current password used in all files is testing!123

ls -a .*.info


rm -rf .GITEA.info  .POST.info  .STEP.info  .TEST.info



Step 5 build and run it


su - dockeruser1

cd /srv/docker-config/gitea-postgres-stepca-docker

./build-it-and-run-it.sh



Step 6 Setup host windows/client computer


Install step-ca root certificate on host windows 


Download the root CA certificate and ignore the certificate with the “-k” option as its will be subsequently installed.


#use hostname and port set in .env for the step-ca service (step 2 above)

curl -k -o stepCA.crt https://newhost-step.rjruss.org:9011/roots.pem



Using Powershell add the downloaded certificate as trusted.


Import-Certificate -FilePath ".\stepCA.crt" -CertStoreLocation cert:\CurrentUser\Root



Answer YES at the prompt


Setup host/dns resolution for Gitea

Update the host file with the Gitea hostname so it resolves.

By default the hostname is newhost-gitea.rjruss.org


Step 7 Install Gitea and register User


Access the Gitea URL newhost-gitea.rjruss.org:3011


Do not change any settings on the Initial Configuration screen but scroll down to the bottom of the page


Hit the “Install Gitea” and wait for the green tea cup to change to the login/register user page


No users have been created for Gitea so register the required user.



Local git setup

  git config --global user.email "Email@used.above"

  git config --global user.name "Username-used-above"




Step 8 Restart the compose project 


By default at initial install Gitea insists on creating the LFS_JWT_SECRET token in the app.ini. By restarting the script will move this out to a dedicated file. This will only occur at the next restart of Gitea. Therefore restart the compose project.



docker compose -f c_docker_compose.yml stop



Output should be as follows with the default configuration

[+] Stopping 3/3

 ✔ Container base-gitea-run   Stopped                                                                                                                                                                   10.3s

 ✔ Container base-postdb-run  Stopped                                                                                                                                                                   10.2s

 ✔ Container base-step-run    Stopped 


The compose file c_docker_compose.yml uses the include notation to add all the individual compose files into one manageable file.


Start again


docker compose -f c_docker_compose.yml start



Output should be as follows with the default configuration

[+] Running 3/3

 ✔ Container base-step-run    Started                                                                                                                                                                    0.3s

 ✔ Container base-postdb-run  Healthy                                                                                                                                                                    5.7s

 ✔ Container base-gitea-run   Started 



Further posts to follow on adapting this Docker repo for other use cases.



Miscellaneous actions


Remove demo certs from certstore, from the example code only - if the root CA name is changed then the *basestep* match won't match any…..


Get-ChildItem -Path Cert:\CurrentUser\Root | Where-Object { $_.Subject -like '*basestep*' } | ForEach-Object {

    certutil -user -delstore Root $_.Thumbprint

}





Saturday, June 24, 2023

OpenSSH on Windows 2019 with Powershell as default ssh shell

 

INSTALL OPENSSH on WINDOWS 2019





Link to online help

https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse?tabs=gui

Install OpenSSH for Windows


Prerequisites

Before you start, your computer must meet the following requirements:


  1. A device running at least Windows Server 2019 or Windows 10 (build 1809).

  2. PowerShell 5.1 or later.

  3. An account that is a member of the built-in Administrators group.


Check I have a admin shell

PS C:\Windows\system32> (New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

True

Current version of powershell

PS C:\Windows\system32> $PSVersionTable.PSVersion


Major  Minor  Build  Revision

-----  -----  -----  --------

5      1      17763  3770


My base level 2019 version

PS C:\Windows\system32> systeminfo |findstr "OS "

OS Name:                   Microsoft Windows Server 2019 Datacenter

OS Version:                10.0.17763 N/A Build 17763


https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-hotfix?view=powershell-7.3


My hot fix levels

PS C:\Windows\system32> Get-HotFix


Source        Description      HotFixID      InstalledBy          InstalledOn

------        -----------      --------      -----------          -----------

W2019A        Update           KB5027131     NT AUTHORITY\SYSTEM  18/06/2023 00:00:00

W2019A        Security Update  KB4512577                          07/09/2019 00:00:00

W2019A        Update           KB4589208     NT AUTHORITY\SYSTEM  18/06/2023 00:00:00

W2019A        Security Update  KB5027222     NT AUTHORITY\SYSTEM  18/06/2023 00:00:00

W2019A        Security Update  KB5023789     NT AUTHORITY\SYSTEM  18/06/2023 00:00:00



PS C:\Windows\system32> wmic qfe list

Caption                                     CSName  Description      FixComments  HotFixID   InstallDate  InstalledBy          InstalledOn  Name  ServicePackInEffect  Status

http://support.microsoft.com/?kbid=5027131  W2019A  Update                        KB5027131               NT AUTHORITY\SYSTEM  6/18/2023

http://support.microsoft.com/?kbid=4512577  W2019A  Security Update               KB4512577                                    9/7/2019

https://support.microsoft.com/help/4589208  W2019A  Update                        KB4589208               NT AUTHORITY\SYSTEM  6/18/2023

https://support.microsoft.com/help/5027222  W2019A  Security Update               KB5027222               NT AUTHORITY\SYSTEM  6/18/2023

                                            W2019A  Security Update               KB5023789               NT AUTHORITY\SYSTEM  6/18/2023


PS C:\Windows\system32>

Check the openssh packages available

PS C:\Windows\system32>  Get-WindowsCapability -Online| out-string  -stream |select-string -pattern "openssh"


Name  : OpenSSH.Client~~~~0.0.1.0

Name  : OpenSSH.Server~~~~0.0.1.0



PS C:\Windows\system32>

Install server & client

PS C:\Windows\system32> Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0

Path          :

Online        : True

RestartNeeded : False


PS C:\Windows\system32> Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0

Path          :

Online        : True

RestartNeeded : False


PS C:\Windows\system32>

Check the status of the new sshd service

PS C:\Windows\system32> Get-Service sshd | Select-Object -Property Name,Status,StartType


Name  Status StartType

----  ------ ---------

sshd Stopped    Manual



PS C:\Windows\system32>

Start it and set to automatic

PS C:\Windows\system32> Start-Service sshd

PS C:\Windows\system32> Set-Service -Name sshd -StartupType 'Automatic'

PS C:\Windows\system32> Get-Service sshd | Select-Object -Property Name,Status,StartType


Name  Status StartType

----  ------ ---------

sshd Running Automatic



PS C:\Windows\system32>

Check Firewall is allowed for ssh

PS C:\Windows\system32> Get-NetFirewallRule -Name *ssh*



Name                  : OpenSSH-Server-In-TCP

DisplayName           : OpenSSH SSH Server (sshd)

Description           : Inbound rule for OpenSSH SSH Server (sshd)

DisplayGroup          : OpenSSH Server

Group                 : OpenSSH Server

Enabled               : True

Profile               : Any

Platform              : {}

Direction             : Inbound

Action                : Allow

EdgeTraversalPolicy   : Block

LooseSourceMapping    : False

LocalOnlyMapping      : False

Owner                 :

PrimaryStatus         : OK

Status                : The rule was parsed successfully from the store. (65536)

EnforcementStatus     : NotApplicable

PolicyStoreSource     : PersistentStore

PolicyStoreSourceType : Local


PS C:\Windows\system32>

Check the env variables

PS C:\Windows\system32> echo $env:ProgramData

C:\ProgramData

PS C:\Windows\system32> dir $env:ProgramData\ssh.


    Directory: C:\ProgramData\ssh


Mode                LastWriteTime         Length Name

----                -------------         ------ ----

d-----       18/06/2023     10:23                logs

-a----       18/06/2023     10:23              6 sshd.pid

-a----       05/09/2018     15:07           2253 sshd_config

-a----       18/06/2023     10:23            672 ssh_host_dsa_key

-a----       18/06/2023     10:23            617 ssh_host_dsa_key.pub

-a----       18/06/2023     10:23            227 ssh_host_ecdsa_key

-a----       18/06/2023     10:23            189 ssh_host_ecdsa_key.pub

-a----       18/06/2023     10:23            419 ssh_host_ed25519_key

-a----       18/06/2023     10:23            109 ssh_host_ed25519_key.pub

-a----       18/06/2023     10:23           1679 ssh_host_rsa_key

-a----       18/06/2023     10:23            409 ssh_host_rsa_key.pub


PS C:\Windows\system32>

Generate new key pair

I chose no password 


My user is robert so you would need to check that :)

PS C:\Windows\system32>  ssh-keygen

Generating public/private rsa key pair.

Enter file in which to save the key (C:\Users\robert/.ssh/id_rsa):

Created directory 'C:\Users\robert/.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in C:\Users\robert/.ssh/id_rsa.

Your public key has been saved in C:\Users\robert/.ssh/id_rsa.pub.

The key fingerprint is:

SHA256:K1R82wY75a/NDK4ltnHmAI7SzCJJNvgCLqNc8hzRaR0 rjruss\robert@w2019a

The key's randomart image is:

+---[RSA 2048]----+

|                 |

|       .         |

|       Eo o .    |

| .  . o... B     |

|o +. +..S + +    |

|o+ oo= o o o .   |

|+o+oo * o = = .  |

|+o=..o . . @ *   |

|.. o      o.+ +  |

+----[SHA256]-----+

PS C:\Windows\system32>

Use Puttygen to generate my client key pair


Follow the prompt to generate randomness

Save public and private keys – you can choose to use a password this will be required 

Export openssh key 

3 files now available on my PC

Copy the public key section as directed by puttygen – we need this on the w2019 server

Paste contents into authorized_keys file in the users .ssh directory (that was created by ssh-keygen) reminder I am using robert as the user - check your user



Check owner

PS C:\Windows\system32> (get-acl \users\robert\.ssh\authorized_keys).owner

RJRUSS\robert

Set the permissions on that file

PS C:\Windows\system32> cd \users\robert\.ssh

PS C:\users\robert\.ssh> (get-acl ~\.ssh\authorized_keys).owner

RJRUSS\robert


PS C:\users\robert\.ssh>  icacls \users\robert\.ssh\authorized_keys

\users\robert\.ssh\authorized_keys NT AUTHORITY\SYSTEM:(F)

                                   BUILTIN\Administrators:(F)

                                   RJRUSS\robert:(F)


Successfully processed 1 files; Failed processing 0 files

PS C:\users\robert\.ssh>

PS C:\users\robert\.ssh> icacls \users\robert\.ssh\authorized_keys /inheritance:r

processed file: \users\robert\.ssh\authorized_keys

Successfully processed 1 files; Failed processing 0 files

PS C:\users\robert\.ssh>

PS C:\users\robert\.ssh>  icacls \users\robert\.ssh\authorized_keys /remove BUILTIN\Administrators

processed file: \users\robert\.ssh\authorized_keys

Successfully processed 1 files; Failed processing 0 files

PS C:\users\robert\.ssh>

Update the sshd config file

Remove the # from these two lines to prevent passwords - this disables passwords and only key based logon allowed

Comment out the match authorized keyfile line


SAVE changes

Restart sshd service

PS C:\users\robert\.ssh> Stop-Service sshd

PS C:\users\robert\.ssh> Start-Service sshd



Setup new putty session

User the user@x.x.x.x format

Add the ppk key in the auth section

Accept new host & Dos prompt should appear

Change to powershell default


Download the required release of powershell

https://github.com/PowerShell/PowerShell/releases




I used the installation version

https://github.com/PowerShell/PowerShell/releases/download/v7.3.4/PowerShell-7.3.4-win-x64.msi





I selected all options you may want to consider deselecting if you don’t need/want the feature

I chose to update via Microsoft update – always worth considering


Launch it to test


Update ssh_config file

Subsystem powershell c:/progra~1/powershell/7/pwsh.exe -sshs -NoLogo

Save

Check the openssh registry details


PS C:\Program Files\PowerShell\7> dir "c:/progra~1/powershell/7/pwsh.exe"


    Directory: C:\Program Files\powershell\7


Mode                 LastWriteTime         Length Name

----                 -------------         ------ ----

-a---          12/04/2023    18:07         293312 pwsh.exe


PS C:\Program Files\PowerShell\7>

Update the registry

PS C:\users\robert\.ssh> New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "c:/progra~1/powershell/7/pwsh.exe" -PropertyType String -Force



DefaultShell : c:/progra~1/powershell/7/pwsh.exe

PSPath       : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SOFTWARE\OpenSSH

PSParentPath : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\SOFTWARE

PSChildName  : OpenSSH

PSDrive      : HKLM

PSProvider   : Microsoft.PowerShell.Core\Registry




PS C:\users\robert\.ssh>

Refresh openssh registry view and default shell appears

Open putty again and connect

Now defaults to powershell

Check if administrator 

PowerShell 7.3.4

PS C:\Users\robert> (New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdent

ity]::GetCurrent())).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

True

PS C:\Users\robert> function Test-Administrator

>> {

>>     $user = [Security.Principal.WindowsIdentity]::GetCurrent();

>>     (New-Object Security.Principal.WindowsPrincipal $user).IsInRole([Security.Principal.WindowsBu

iltinRole]::Administrator)

>> }

PS C:\Users\robert> Test-Administrator

True















Google +