Go back to the main page

SAP Skills Portal

Server Administration of the SAP Specialist Web Portal
Server Details

Web Server

PropertyValue
Hostnamesapidesecc8.fivetran-internal-sales.com
Operating SystemSUSE Linux Enterprise Server 15 SP5
Python/root/miniconda/bin/python3 (3.13.9)
Web Port (Production)443 (HTTPS)
Web Port (Dev)8443 (HTTPS)
Server Processgcs_explorer_server.py
Static Files/usr/sap/sap_skills/
Portal URLhttps://sapidesecc8.fivetran-internal-sales.com/sap_skills/
Important: Always use /root/miniconda/bin/python3, NOT /usr/bin/python3 (the system Python lacks duckdb/pyarrow).
SSH Access

Common SSH Commands

ActionCommand
Check server statusssh root@sapidesecc8 "systemctl status gcs-explorer --no-pager"
Restart productionssh root@sapidesecc8 "systemctl restart gcs-explorer"
Restart devssh root@sapidesecc8 "systemctl restart gcs-explorer-dev"
View logsssh root@sapidesecc8 "tail -40 /var/log/gcs_explorer.log"
List portal filesssh root@sapidesecc8 "ls -la /usr/sap/sap_skills/"
Check disk spacessh root@sapidesecc8 "df -h /"
Check cron jobsssh root@sapidesecc8 "crontab -l"
Architecture

How It Works

The SAP Skills Portal is a set of static HTML files served by the GCS Parquet Explorer web server. The server is a Python HTTPS application running on port 443.

Browser
  |
  | HTTPS (port 443)
  v
sapidesecc8 (gcs_explorer_server.py)
  |
  |-- /sap_skills/          --> /usr/sap/sap_skills/index.html (static)
  |-- /sap_skills/docs/*    --> /usr/sap/sap_skills/docs/*.html (static)
  |-- /sap_skills/downloads --> /usr/sap/sap_skills/downloads/ (JCo JARs)
  |-- /datalake_reader/     --> GCS Parquet Explorer (dynamic, auth required)

Directory Structure

/usr/sap/sap_skills/
  index.html                                        # Landing page
  docs/
    SAP_OTC_Workflow_Documentation.html              # Order to Cash
    SAP_P2P_Workflow_Documentation.html              # Procure to Pay
    SAP_PP_Workflow_Documentation.html               # Plan to Produce
    SAP_MRP_Workflow_Documentation.html              # Material Requirements Planning
    SAP_Housekeeping_Documentation.html              # Housekeeping & Job Scheduler
    SAP_CDS_Workflow_Documentation.html              # CDS View Extraction
    SAP_GCS_Explorer_Documentation.html              # GCS Parquet Explorer
    SAP_Skills_Portal_Documentation.html             # Portal landing page copy
    SAP_Skills_Portal_Server_Documentation.html      # This document
  downloads/
    sapjco3.jar                                      # SAP JCo library
    libsapjco3.dylib                                 # JCo native (macOS)
    gson.jar                                         # Google Gson

Static File Route

The /sap_skills/ route is handled inside gcs_explorer_server.py in the _do_GET() method. It:

  • Redirects /sap_skills to /sap_skills/
  • Maps /sap_skills/ to /usr/sap/sap_skills/index.html
  • Serves any file under /usr/sap/sap_skills/ with correct MIME types
  • Blocks path traversal (.. in path)
  • Requires no authentication (public access)
Warning: If gcs_explorer_server.py is replaced with a version that lacks this route, the portal will return 404. Always verify the route exists after server.py updates.
SSL Certificate

Current Certificate

PropertyValue
IssuerZeroSSL RSA Domain Secure Site CA
SubjectCN=sapidesecc8.fivetran-internal-sales.com
Valid FromMar 27, 2026
ExpiresJun 25, 2026
Cert File/usr/sap/gcs_explorer_cert.pem (chained: server + intermediate CA)
Key File/usr/sap/gcs_explorer_key.pem

Check Expiry

ssh root@sapidesecc8 "openssl x509 -in /usr/sap/gcs_explorer_cert.pem -noout -dates"

Renewal Procedure

1Renew at ZeroSSL

Go to zerossl.com and renew the certificate for sapidesecc8.fivetran-internal-sales.com. Download the new certificate and key files.

2Upload New Files

scp new_cert.pem root@sapidesecc8:/usr/sap/gcs_explorer_cert.pem
scp new_key.pem root@sapidesecc8:/usr/sap/gcs_explorer_key.pem
ssh root@sapidesecc8 "chmod 600 /usr/sap/gcs_explorer_key.pem"

3Restart Services

ssh root@sapidesecc8 "systemctl restart gcs-explorer; systemctl restart gcs-explorer-dev"
Chain the certificate: The PEM must contain both the server cert and intermediate CA. If ZeroSSL provides them separately:
cat server.crt intermediate.crt > /usr/sap/gcs_explorer_cert.pem
Resilience & Auto-Recovery

Systemd Services

ServicePurposePortRestart Policy
gcs-explorer.serviceProduction web server443Restart=always, RestartSec=5
gcs-explorer-dev.serviceDev web server8443Restart=always, RestartSec=5
gcs-explorer-watchdog.timerHealth checkEvery 60 seconds

All three are enabled — they start automatically after a server reboot.

Auto-Restart on Crash

Both server services have Restart=always with RestartSec=5. If the Python process crashes, systemd restarts it within 5 seconds.

# Simulate a crash and verify auto-restart
ssh root@sapidesecc8 "systemctl kill -s SIGKILL gcs-explorer; sleep 6; systemctl status gcs-explorer --no-pager"
# Should show "active (running)" with a recent start time

Health Watchdog

The watchdog timer runs every 60 seconds (first run 120 seconds after boot). It:

  1. Sends a health check to https://localhost/datalake_reader/api/health
  2. If no response within 10 seconds, restarts the production service
  3. Does the same for the dev service on port 8443
  4. Logs all restarts to /var/log/gcs_explorer_watchdog.log
# Check watchdog status and recent activity
ssh root@sapidesecc8 "systemctl status gcs-explorer-watchdog.timer --no-pager"
ssh root@sapidesecc8 "tail -10 /var/log/gcs_explorer_watchdog.log"

Verify All Resilience

# All three should show "enabled"
ssh root@sapidesecc8 "systemctl is-enabled gcs-explorer gcs-explorer-dev gcs-explorer-watchdog.timer"

# Test HTTPS endpoints
curl -sk https://sapidesecc8.fivetran-internal-sales.com/datalake_reader/api/health
curl -sk -o /dev/null -w "%{http_code}" https://sapidesecc8.fivetran-internal-sales.com/sap_skills/
Backup

Scheduled Backup

PropertyValue
ScheduleDaily at 23:00, Monday–Friday (weekends excluded)
Mechanismcron on sapidesecc8
Script/usr/local/bin/backup_webserver.sh
Log/var/log/webserver_backup.log
GCS Bucketgs://sap-hana-backint/sapidesecc8_webserver/
Backup Size~9 MB (21 files)

Cron Entry

# Runs Mon-Fri at 23:00
0 23 * * 1-5 /usr/local/bin/backup_webserver.sh

Verify the cron is active:

ssh root@sapidesecc8 "crontab -l"

What Gets Backed Up

gs://sap-hana-backint/sapidesecc8_webserver/
  gcs_explorer_server.py          # Production web server (port 443)
  gcs_explorer_server_dev.py      # Dev web server (port 8443)
  gcs_explorer_cert.pem           # SSL certificate (ZeroSSL, chained)
  gcs_explorer_key.pem            # SSL private key
  gcs_explorer.env                # Environment variables (Polaris credentials)
  RESTORE_GUIDE.md                # Disaster recovery guide
  sap_skills/
    index.html                    # SAP Skills Portal landing page
    docs/                         # All HTML documentation pages
    downloads/                    # sapjco3.jar, libsapjco3.dylib, gson.jar
  systemd/
    gcs-explorer.service          # Production systemd unit
    gcs-explorer-dev.service      # Dev systemd unit
    gcs-explorer-watchdog.service # Health check watchdog

Manual Backup

# Run backup immediately
ssh root@sapidesecc8 "/usr/local/bin/backup_webserver.sh"

# Check the log
ssh root@sapidesecc8 "tail -5 /var/log/webserver_backup.log"

# Verify bucket contents
gsutil ls -r gs://sap-hana-backint/sapidesecc8_webserver/
gsutil du -sh gs://sap-hana-backint/sapidesecc8_webserver/
Disaster Recovery — Full Restore
When to use: If the server is rebuilt, the web server files are lost, or the portal stops working after a server.py replacement.

Step-by-Step Restore

1Download Backup from GCS

ssh root@sapidesecc8
mkdir -p /tmp/restore
gsutil -m rsync -r gs://sap-hana-backint/sapidesecc8_webserver/ /tmp/restore/

2Restore Web Server Files

# Python server files
cp /tmp/restore/gcs_explorer_server.py /usr/sap/gcs_explorer_server.py
cp /tmp/restore/gcs_explorer_server_dev.py /usr/sap/gcs_explorer_server_dev.py

# SSL certificate and key
cp /tmp/restore/gcs_explorer_cert.pem /usr/sap/gcs_explorer_cert.pem
cp /tmp/restore/gcs_explorer_key.pem /usr/sap/gcs_explorer_key.pem
chmod 600 /usr/sap/gcs_explorer_key.pem

# Environment variables (Polaris OAuth credentials)
cp /tmp/restore/gcs_explorer.env /usr/sap/gcs_explorer.env
chmod 600 /usr/sap/gcs_explorer.env

3Restore SAP Skills Portal

# Create directory structure
mkdir -p /usr/sap/sap_skills/docs
mkdir -p /usr/sap/sap_skills/downloads

# Copy portal files
cp /tmp/restore/sap_skills/index.html /usr/sap/sap_skills/
cp /tmp/restore/sap_skills/docs/*.html /usr/sap/sap_skills/docs/
cp /tmp/restore/sap_skills/downloads/* /usr/sap/sap_skills/downloads/

4Install Systemd Services

# Copy service files
cp /tmp/restore/systemd/gcs-explorer.service /etc/systemd/system/
cp /tmp/restore/systemd/gcs-explorer-dev.service /etc/systemd/system/
cp /tmp/restore/systemd/gcs-explorer-watchdog.service /etc/systemd/system/

# Recreate the watchdog timer (not in backup)
cat > /etc/systemd/system/gcs-explorer-watchdog.timer << 'EOF'
[Unit]
Description=Run GCS Explorer Health Watchdog every 60s

[Timer]
OnBootSec=120
OnUnitActiveSec=60
AccuracySec=5

[Install]
WantedBy=timers.target
EOF

# Reload, enable, and start everything
systemctl daemon-reload
systemctl enable gcs-explorer gcs-explorer-dev gcs-explorer-watchdog.timer
systemctl start gcs-explorer gcs-explorer-dev gcs-explorer-watchdog.timer

5Restore Cron Backup Schedule

# Re-add the daily backup cron (Mon-Fri at 23:00)
(crontab -l 2>/dev/null; echo "0 23 * * 1-5 /usr/local/bin/backup_webserver.sh") | crontab -

# Verify
crontab -l

6Verify Everything

# Check services are active
systemctl status gcs-explorer --no-pager
systemctl status gcs-explorer-dev --no-pager
systemctl status gcs-explorer-watchdog.timer --no-pager

# Test HTTPS endpoints
curl -sk https://localhost/datalake_reader/api/health
curl -sk https://localhost:8443/datalake_reader_dev/api/health
curl -sk -o /dev/null -w "%{http_code}" https://localhost/sap_skills/

# Check logs for errors
tail -20 /var/log/gcs_explorer.log
Deployment

Update Landing Page

# Edit locally, then upload (no restart needed)
scp /Users/antonio.carbone/SAP_Skills/index.html root@sapidesecc8:/usr/sap/sap_skills/index.html

Static files are served directly from disk — no server restart or cache clearing needed.

Add/Update a Documentation Page

# Upload a new or updated doc
scp /Users/antonio.carbone/SAP_Skills/docs/NEW_DOC.html \
    root@sapidesecc8:/usr/sap/sap_skills/docs/

Sync Portal Documentation Copy

# Keep the documentation copy in sync with the landing page
cp /Users/antonio.carbone/SAP_Skills/index.html \
   /Users/antonio.carbone/SAP_Skills/docs/SAP_Skills_Portal_Documentation.html

Update GitHub Repository

cd /tmp/SAPSkillsPortal
cp /Users/antonio.carbone/SAP_Skills/index.html .
cp /Users/antonio.carbone/SAP_Skills/docs/*.html docs/
cp /Users/antonio.carbone/.claude/commands/SAP_Skills/sap-skills-portal.md skill/
git add -A && git commit -m "Update portal" && git push
Troubleshooting
IssueFix
Portal returns 404Check /sap_skills/ route exists in gcs_explorer_server.py _do_GET()
Server not runningssh root@sapidesecc8 "systemctl restart gcs-explorer"
Doc page not loadingssh root@sapidesecc8 "ls /usr/sap/sap_skills/docs/"
After server.py updateRe-add /sap_skills/ route block to _do_GET()
SSL certificate expiredRenew at zerossl.com, upload new files, restart services (see SSL section)
Backup not runningssh root@sapidesecc8 "crontab -l" — verify cron entry exists
Watchdog not restartingssh root@sapidesecc8 "systemctl status gcs-explorer-watchdog.timer --no-pager"
Changes not visibleFiles served from disk — no cache. Hard-refresh browser (Cmd+Shift+R).
Python import errorsUse /root/miniconda/bin/python3, not /usr/bin/python3
Backup cron missing after restoreRe-add: (crontab -l; echo "0 23 * * 1-5 /usr/local/bin/backup_webserver.sh") | crontab -
Email Relay

SMTP Configuration

The web server (sapidesecc8) sends email via smtp2go relay using Postfix. The SAP Skills Portal uses sap-skills@fivetran-internal-sales.com as the from address.

PropertyValue
Providersmtp2go
SMTP Servermail.smtp2go.com:2525
Portal Fromsap-skills@fivetran-internal-sales.com
Server Fromsapidesecc8@fivetran-internal-sales.com
Verified Domainfivetran-internal-sales.com
Vault Keysmtp2go
Config Files/etc/postfix/main.cf, /etc/postfix/sasl_passwd

All Linux servers in the landscape are configured with the same smtp2go relay. Each uses <hostname>@fivetran-internal-sales.com as its from address.

Mailing Lists

Distribution Lists

Mailing lists used for alerts, notifications, and reports from the SAP landscape. Lists are stored in this page and can be read by the server APIs.

SAPSpecialists

Alerts, notifications, reports from SAP landscape
How it works: Changes are saved to this HTML file on the server via the /sap_skills/api/update_mailing_list API. The server reads the distribution list from this page when sending emails.