Cloud Sites Timeout on CMS Install
Timeout message

Timeout message

Load balancers usually have a limit on how long a TCP connection will remain open to a web server before timing out on the connection. With Cloud Sites in particular, the load balancer will time out on an HTTP response if the PHP code does not send output to the browser for 30 seconds. After this timeout, the load balancer will send its own HTML error page instead of the PHP output. The code will continue to run and generate its HTML output, but the HTML will not be sent to the browser as the TCP connection was already closed when the browser was presented with the error message.

A web page will normally load in much less than 30 seconds as websites that take 30 seconds to load a page would be nearly unsurfable, so this timeout rarely becomes an issue, however some administrative tasks may take longer than this to complete.  One case where this is particularly obtrusive is with some install wizards.  During installation, it is not uncommon to have the PHP for a page perform a number of time intensive tasks totaling to over 30 seconds.  Some CMS installers will resume by refreshing the page, though most will start over. If the operation just needs to run once, you may consider running the install on a server and then migrating the installed site to the production environment, however this isn’t always an option.

To have a page run for longer than 30 seconds, data must be occasionally sent back to the load balancer.  Normally the PHP interpreter will store the HTML output of the PHP script in a buffer until the script is completed, but you can send the contents of the buffer to the browser using the PHP flush() function.

The following script runs for 45 seconds and keeps the connection open by flushing its output to the browser, though this may not be noticed within the browser itself as the browser would typically buffer as well.

 

if (ob_get_level()==0) ob_start();
for ( $i = 0; $i < 45; $i++) {
    print($i."\n");
    ob_flush();
    flush();
    sleep(1);
    } 

By inserting print(" "), ob_flush(), and flush() to a php script, the timeout can be avoided.

With Prestashop 1.5.4.1 in particular, the installer usually stops at the database creation and language installation setting from a JQuery HTTP request taking longer than 30 seconds to complete.  installDefaultData() in install/models/install.php is the primary function to handle this step of the install process.  To prolong the timeout, all that is needed is to throw a few print() and flush() commands into this function and its subroutines createShop() and installLanguages() like so.

 

...
167         public function installDefaultData($shop_name, $clear_database = false)
168         {
169                 ob_end_flush();
170                 print(" ");
171                 flush();
...
252         public function createShop($shop_name)
253         {
254                 // Create default group shop
255                 print(" ");
256                 flush();
...
267                 // Create default shop
268                 print(" ");
269                 flush();
...
283                 print(" ");
284                 flush();
285 
286                 Context::getContext()->shop = $shop;
...
297                 print(" ");
298                 flush();
299 
300                 if (!$shop_url->add())
...
320                 foreach ($languages_list as $iso)
321                 {
322                         print(" ");
323                         flush();
...

These commands will occasionally print whitespace to the JSON output which are ignored by the browser, but keep the load balancer from timing out.

No Comments Posted in Uncategorized
Tagged , , , , , ,
Rackspace Cloud DNS API Interface

dnspanel.zip

This zip file contains a php web application to give a GUI interface to the Rackspace Cloud DNS API. It requires Linux, PHP, and curl be installed on the server it runs on. Just unzip the files and upload them to a web server, then hit them with your browser to get started.

I have built this DNS interface mostly using these two links:
http://www.collazo.ws/2011/07/08/using-rackspace-dnsaas-with-curl-part-1
http://docs.rackspace.com/api/

Once you log in, the application gives you the chance to designate a subaccount number. This is useful if using the Clients feature of the Cloud Sites service.

Enjoy!

DNS API Record List Screenshot

7 Comments Posted in Uncategorized
Limiting a Cron Job to a specific day of the week in Cloud Sites

The Cloud Sites control panel does not have a built in option to run a cron job only on a certain day of the week. There are cases where you would like a job to only run on a certain weekday such as a weekly mailout or a weekend maintenance script.

There is an option to run every few days, however this option, similar to the cron Linux option it calls to, would work by finding the remainder of the day of the month and dividing it by the number you provide then run the job if this remainder equals 1. So if this was set to run every 7 days and the first day of the month is a Wednesday, then the job will run on every Wednesday of the current month, however it may run on any other day of the week next month depending on which day of the week the next month started on.

In order to run a cron job on a certain day of the week, you can run a check at the beginning of the script to check the day of the week and exit if it is not the day that the job should run on.

The following line can be appended to the top of a bash script for it to check if the day of the week is Sunday and exit if it is not. That way you can run the script every day of the week and it will only actually complete on the day that you designate.

if [[ -z "`date '+%a' | grep Sun`" ]]; then exit; fi

No Comments Posted in Uncategorized
Listing files modified on a certain day

You can retrieve a list of files that were last modified yesterday in a wide variety of ways with scripting languages. The “find” command would probably be the simplest method in Linux by using the mtime option like so:

#find -maxdepth 1 -mtime +1 -mtime -2

The find command is disabled however in Cloud Sites for performance reasons, so an alternative must be found. Here is one example of a line that can be run in a .sh file that can also give a list of files modified yesterday.

ls -l --time-style='+%F' /mnt/stor15-wc2-dfw1/123456/backups/ | awk '{print $6,$7}' | grep "`date --date='1 day ago' '+%F'`" | awk '{print $2}'

The directory path must be replaced by your full server side path before using this command.

This command can be run through a cron job within a .sh file in the same manner that unzip commands are run through a .sh file in this article:
http://cloudsites.rackspacecloud.com/index.php/Use_cron_to_extract_and_compress_%28zip_%26_unzip%29

This bash one liner will first list the contents of a directory putting the time in the %F format (consult documentation on the date command), cuts out all columns save for the date and filename, only displays those lines with yesterday’s date within them, and then displays just the filename column. The output of this command will be a list of filenames, one per line, of files within this directory that were modified within the last day which can then be processed using an xargs command or piped into a while loop like so:

ls -l --time-style='+%F' /full/path/backups/ | awk '{print $6,$7}' | grep "`date --date='1 day ago' '+%F'`" | awk '{print $2}' | while read line; do rm /full/path/$line; done

More information on modifying the day to check for using the date command can be found here:
http://ss64.com/bash/date.html

No Comments Posted in Uncategorized
Unzipping files through php

The following script will unzip all files within the current directory and then print its results to the browser. To unzip a file, upload your .zip files and then upload this php file to the same directory and then visit this page in your browser. You could for example save this file as unzip.php and then upload it through FTP to /www.donotexec.com/web/content/ and then run it by visiting http://www.donotexec.com/unzip.php in your browser.

Happy coding!

1
2
3
4
<?php
$output = shell_exec('ls | grep .zip$ | while read line; do unzip `pwd`/$line; done 2>&1');
echo "<pre>$output</pre".">";
?>
No Comments Posted in php
Tagged , , ,
Allow or Deny by IP in IIS Cloud Sites

The steps:
Add the following entry into your web.config file and then place a SecurityHttpModule.cs file within a folder named App_Code with the code given. Replace 127.0.0.1 with the IP address that you would like to allow for your domain.
Any questions? Feel free to ask! But be sure to be specific so I can tell your comment is not spam!

My Sources:
Original code from:

The following used to aid in implementation:

WARNINGS:
This code works on my sites, however you may not be able to implement this on your own site. I am not responsible for any damages or data loss that may result in implementing this code. As always, whenever changing the configuration or modifying code on a site, test the code in a staging environment first and back up all data including any databases before implementing.

Applying this code to an entire site will cause compute cycles to rise significantly as every http request will rely on c# code to be run even for static files such as txt, html, and css. For this reason I would recommend not using this on a site that receives a considerable number of hits and to isolate protected content within its own subdomain. Consider carefully if static files, especially those served by the Media Accelerator, would need to run through this screening process and if not, host them on a separate site to reduce the compute cycles consumed.

Add to Web.Config:

1
2
3
4
5
6
7
8
<?xml version="1.0"?>
<configuration>
    <system.webServer>
        <modules>
            <add name="SecurityHttpModule" type="SecurityHttpModule"/>
        </modules>
    </system.webServer>
</configuration>

Save in App_Code\SecurityHttpModule.cs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
///
/// HTTP module to restrict access by IP address
/// From http://www.codeproject.com/KB/aspnet/http-module-ip-security.aspx
/// Modified with:
///  http://msdn.microsoft.com/en-us/ms227673.aspx
///  http://www.geekpedia.com/KB84_The-name-Request-does-not-exist-in-the-current-context.html
///  http://cloudsites.rackspacecloud.com/index.php/How_do_I_deny_certain_IP_addresses_from_accessing_my_site%3F
///
using System;
using System.Web;
public class SecurityHttpModule: IHttpModule
{
 public SecurityHttpModule() { }
    public void Init(HttpApplication context)
    {
        context.BeginRequest += new EventHandler(Application_BeginRequest);
    }
    private void Application_BeginRequest(object source, EventArgs e)
    {
        HttpContext context = ((HttpApplication)source).Context;
        string ipAddress =  System.Web.HttpContext.Current.Request.ServerVariables["HTTP_X_CLUSTER_CLIENT_IP"];
		string path = ((HttpApplication)source).Request.Path;
		string ext = path.Substring(path.Length-3, 3).ToLower();
        if (!IsValidIpAddress(ipAddress) &amp;&amp; ext != "jpg" &amp;&amp; ext != "gif" &amp;&amp; ext != "png" &amp;&amp; ext != "mp3" &amp;&amp; ext != "wav" &amp;&amp; ext != "wma" &amp;&amp; ext != "swf")
        {
            context.Response.StatusCode = 403;  // (Forbidden)
	}
    }
    private bool IsValidIpAddress(string ipAddress)
    {
        return (ipAddress == "127.0.0.1");
    }
    public void Dispose() { /* clean up */ }
}
No Comments Posted in ASP.NET
Tagged , ,
Changing default FTP permissions in Cloud Sites

Setting the default permissions for newly uploaded files is often useful as a convenience, especially when having multiple FTP users within the same site as if all files are owned by the same user, a recursive chmod can be used on the content folder. To allow all FTP users for your site to be able to modify all files, group write permissions would need to be added to these files after they are uploaded. This can usually be done after uploading by right-clicking on the file and selecting that write permissions be added. Some FTP clients can have this chmod operation automated though so that any file that is uploaded is automatically given the permissions that you desire.
Within WinSCP this option can be found under Preferences menu->Preferences… button->Transfer menu under the Set permissions option of the Upload options section as can be seen here.  The option in particular that we are concerned with is W under Group.  For more on file permissions on Cloud Sites, check out this article in the Knowledge Base.

WinSCP default permissions section

In order to avoid most difficulties with using several FTP users, I would suggest limiting an FTP user to one directory and then only using this one FTP user within this directory.

No Comments Posted in FTP
Tagged , ,
Sample ASP.NET code that works in Cloud Sites

Uploading the following 3 files will result in a bare bones ASP.NET site that functions within Rackspace Cloud Sites. It would be possible to trim these down further, especially the web.config, however this is what I came up with through Visual Studio with minimal editing to the resultant files.

web.config:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<?xml version=”1.0″?>
<configuration>
<appSettings/>
<connectionStrings/>
<system.web>
<compilation debug=”true” strict=”true” explicit=”true” />
<pages>
<namespaces>
<clear />
<add namespace=”System” />
<add namespace=”System.Collections” />
<add namespace=”System.Collections.Specialized” />
<add namespace=”System.Configuration” />
<add namespace=”System.Text” />
<add namespace=”System.Text.RegularExpressions” />
<add namespace=”System.Web” />
<add namespace=”System.Web.Caching” />
<add namespace=”System.Web.SessionState” />
<add namespace=”System.Web.Security” />
<add namespace=”System.Web.Profile” />
<add namespace=”System.Web.UI” />
<add namespace=”System.Web.UI.WebControls” />
<add namespace=”System.Web.UI.WebControls.WebParts” />
<add namespace=”System.Web.UI.HtmlControls” />
</namespaces>
</pages>
<authentication mode=”Windows” />
<customErrors mode=”Off”/>
</system.web>
</configuration>

Testing.aspx:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<%@ Page Language="VB" AutoEventWireup="False" CodeFile="Testing.aspx.vb" Inherits="Testing" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
	<title>Testing</title>
</head>
<body>
	<form id="form1" runat="server">
		<div>
			<asp:Label id="lbl1" runat="server" Text="Hello World"/>
		</div>
	</form>
</body>
</html>

Testing.aspx.vb:

1
2
3
4
5
6
7
8
9
10
11
12
'Option Explicit On
'Option Strict On
 
Imports System
Imports System.Web
Imports System.Web.UI
Partial Public Class Testing
	Inherits System.Web.UI.Page
	Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
		lbl1.Text="Hi World!"
	End Sub
End Class
No Comments Posted in ASP.NET
Tagged , ,
Sorting files by datestamp in Bash

You would think there were a function in Linux already that would do this.  The following command will sort all files in a directory and all subdirectories for the past 14 days by datestamp.

bash$ find . -mtime -14 -type f | while read line; do echo -n `ls -l --time-style='+%s %F %T' "$line" | awk '{print $6,$7,$8}'`; echo " $line"; done | sort | cut -d ' ' -f 1 --complement
No Comments Posted in bash, Security
Tagged , ,
Securing WordPress on a shared host

The following articles contain details on securing your WordPress installation on a shared hosting service, specifically Rackspace Cloud Sites.  This post will change as more resources are compiled:

On a shared hosting service running suexec, there should be no reason to have global permissions on a php site:

Backup, Backup, Backup!  Just 2 of many possible backup scripts.

If the unfortunate does occur, these three links contain information on recovering. Due to the possibility of backdoors being injected within your sites though, the safest option would be to restore from one of your backups. You do keep backups, right?

No Comments Posted in Security
Tagged , ,