Improve Page Performance, save PHP array on server?

Question

is it possible to store a PHP-array to my server, right now it always gets created when someone reloads the page from a CSV file but that is unnecessary since the file only chances after each hour.

ATM, the page takes like 9 seconds to load, which is quite long. The CSV file has 10k+ rows with 9 elements per row, so it would be really good for performance if the server didn't have to process 100k elements for each user.

I already have a cronjob for downloading the csv file so it would be good if the parse command would be executed after the download finished, only once per hour.

cronjob:

<?php

function download_remote_file($file_url, $save_to) {
  $content = file_get_contents($file_url);
  file_put_contents($save_to, $content);
}
download_remote_file(<url here>, realpath(".") . '/dump.csv');
?>

and this happens with every reload of the page:

1st: Parse data to array

$url = 'dump.csv';
$csvData = file_get_contents($url);
$lines = explode(PHP_EOL, $csvData);
$array = array();
foreach ($lines as $line) {
$line = str_replace("\\", "&#92;", $line);
$line = str_replace("#", "&#35;", $line);
$array[] = str_getcsv($line);

2nd: pass array to Javascript

var array = <?php echo json_encode( $array ) ?>;

3rd: create HTML table

//some code

4th: initialise data table plugin

$(document).ready( function () {
    createtable();
    $('#scoreboard').DataTable( {
        "iDisplayLength": 50,
        language: {
            decimal: ".",       
        },
        "lengthMenu": false,
        "bLengthChange": false
    } );
} );

Is there something that could be done faster?

Like, as mentioned, save the php array server-side or maybe saving the JS array with the HTML table somehow?

-Innerwolf

You could try storing the parsed array in the session or Memcached. Maybe stored a PHP file with the hard coded array. Open the CSV file, parse it, save it to a PHP file. Then load the PHP file. — Greg Burghardt, Jun 30 '15 at 15:00
I think a database was unnecessary since it also works with php. i also don't know how to store a file from an url directly to sql — Innerwolf, Jun 30 '15 at 15:04
you could simply write a PHP file, containing the generated array, and include this file. Or you could write the json to a javaScript file and include that into your output. — Burki, Jun 30 '15 at 15:10
Why does it need to be in a CSV file and not a Database to begin with? — , Jun 30 '15 at 15:14

score 1 · Answer 1 · answered Jun 30 '15 at 15:13

After you parse your CSV, do this:

$file = fopen('/tmp/output.js', 'w');
fwrite($file, '<script type="text/javascript">');
fwrite($file, 'var array =');
fwrite($file, json_encode( $array ));
fwrite($file, ';');
fwrite($file, '</script>');
fclose($file);

copy('/path/to/script.js', '/path/to/script.js.bak');
move('/tmp/output.js', '/path/to/script.js');

Then, later on when you are outputting the HTML, you just need to stick in a:

<script type="text/javascript" src="/scripts/script.js">

in the header. People's browsers should cache it properly too. Note the copy and move -- you don't strictly need to make a backup copy, but you MUST use a move() to replace the 'live' script -- move() is atomic, more or less, and won't result in anyone getting a half-file.

Also, note that you'll need write permissions to where the script is -- there are ways to keep this pretty secure (not letting your PHP script write all over the hard drive), but that's out of scope here.

score 0 · Accepted Answer · answered Jun 30 '15 at 16:50

Since you mention getting the data on an hourly basis I suggest the following:

grab the CSV file with cron and store the data in a database on an hourly basis
configure your data tables component to use server side data

This way you won't force every user to download the entire array at once on every first page load. The server side script only fetches the number of records that need to be displayed on that particular page in the table.

Improve Page Performance, save PHP array on server?

2 Answers2

Linked