Conversion of 3 .txt files to Tabular form

Hi All,
(1). Please can anyone help me to use PHP or MySQL to automate the conversion and merging of the attached .txt files into a tabular format? I will paste the data into the quote below since I cannot attach the file here.
file 1 is named 2021_11_30.txt and the data there is below :

`----------NEW TRANSACTON-----------
TRANSACTION DATE : 08:42 AM, 30 Nov 2021
CLIENT ID : C00001
CLIENT NAME : SALISU BISI
AMOUNT : 16,450.00
TRANSACTION REF. : 00001

----------NEW TRANSACTON-----------
TRANSACTION DATE : 08:46 AM, 30 Nov 2021
CLIENT ID : C00002
CLIENT NAME : SULE YAYA
AMOUNT : 80,940.00
TRANSACTION REF. : 00002

----------NEW TRANSACTON-----------
TRANSACTION DATE : 08:51 AM, 30 Nov 2021
CLIENT ID : C00001
CLIENT NAME : SALISU BISI
AMOUNT : 44,900.00
TRANSACTION REF. : 00003

----------NEW TRANSACTON-----------
TRANSACTION DATE : 08:58 AM, 30 Nov 2021
CLIENT ID : C00003
CLIENT NAME : ADEGOKE BAYO
AMOUNT : 60,000.00
TRANSACTION REF. : 00004

----------NEW TRANSACTON-----------
TRANSACTION DATE : 09:12 AM, 30 Nov 2021
CLIENT ID : C00004
CLIENT NAME : COKER PRIYE
AMOUNT : 80,200.00
TRANSACTION REF. : 00005
`

file 2 is 2021_12_04.txt

----------NEW TRANSACTON-----------
TRANSACTION DATE : 09:42 AM, 04 Dec 2021
CLIENT ID : C00011
CLIENT NAME : SAINT OBI
AMOUNT : 100,450.00
TRANSACTION REF. : 00101

----------NEW TRANSACTON-----------
TRANSACTION DATE : 09:54 AM, 04 Dec 2021
CLIENT ID : C00112
CLIENT NAME : DESIREE MONICA
AMOUNT : 180,940.00
TRANSACTION REF. : 00102

----------NEW TRANSACTON-----------
TRANSACTION DATE : 09:58 AM, 04 Dec 2021
CLIENT ID : C00211
CLIENT NAME : ALIU MUSA
AMOUNT : 544,000.00
TRANSACTION REF. : 00103

----------NEW TRANSACTON-----------
TRANSACTION DATE : 10:40 AM, 04 Dec 2021
CLIENT ID : C00112
CLIENT NAME : DESIREE MONICA
AMOUNT : 860,000.00
TRANSACTION REF. : 00104

Blockquote

File 3. 2021_12_05.txt

`----------NEW TRANSACTON-----------
TRANSACTION DATE : 09:42 AM, 05 Dec 2021
CLIENT ID : C05011
CLIENT NAME : SAINT JOE PATRICK
AMOUNT : 700,550.00
TRANSACTION REF. : 02101

----------NEW TRANSACTON-----------
TRANSACTION DATE : 10:02 AM, 05 Dec 2021
CLIENT ID : C40112
CLIENT NAME : ABU SALE
AMOUNT : 180,650.00
TRANSACTION REF. : 02102
`

Please, how can I use PHP to convert the 3 files tabular form?

What have you tried so far, and what problems did you have with it? I don’t imagine you’re expecting someone to just sit down and write your code for you.

If your post contains the full data, it’s hardly worth writing any code to import it, just type it in.

When you say “convert into a tabular format”, do you mean that you’re trying to display them in a table, or insert them into a database table?

2 Likes

There are a good number of things to do in order to get your txt files into a usable format, which could then be displayed or uploaded into a database table. As the dates on these files are old I assume you have an archive of old data saved as txt files you would like to get them into a more usable format. Let’s start with some basics.

A very basic form might have a single text input and a submit button. The inputs have both a type and name attributes that could be defined like so.

<form action="" method="post">
	<input type="text" name="name">
	<input type="submit" name="submit_name" value="Submit">
</form>

Now if I wanted to see what is sent when I submit the form I could define a controlling IF condition and print_r() the post.

<?php
if($_SERVER['REQUEST_METHOD'] === 'POST' && !empty($_POST['name'])):
	echo "<pre>";
	print_r($_POST);
	echo "</pre>";	
endif;
?>

and we might see something like this.

(
    [name] => Drummin
    [submit_name] => Submit
)

See how the name attribute in the form become the [KEY] in the $_POST array with the matching => VALUE.

Now to upload a file you would need to go about things a little different. The most obvious change might be to change the input type to type="file"(instead of type="text" in the sample form) but file uploads are sent with multiple attributes and in order for this to happen you need add an enctype attribute to your <form> tag as shown in this example.

<form action="" method="post" enctype="multipart/form-data">
	<input type="file" name="filename">
	<input type="submit" name="submit_file" value="Upload">
</form>

To see what is being sent you can change your IF condition and print_r() to represent this new form.

if($_SERVER['REQUEST_METHOD'] === 'POST' && !empty($_FILES)):
	echo "<pre>";
	print_r($_FILES);
	echo "</pre>";	
endif;

Notice again that the name attribute becomes the primary array KEY and the multiple [KEY] => VALUE attributes related to the file upload. It might look something like this.

Array
(
    [filename] => Array
        (
            [name] => 2021_12_04.txt
            [type] => text/plain
            [tmp_name] => user_path_to\Temp\php1E5A.tmp
            [error] => 0
            [size] => 674
        )
)

Now that we have some of the basics out of the way let’s move forward. As you want to upload several files at once we need our file input to be able to accept multiple files. Start by adding square brackets after the name of your input and the term multiple to your input.

<input type="file" name="filename[]" multiple>

Now while holding down Ctrl on your keyboard you can load several files into your input which will look like this when sent.

Array
(
    [filename] => Array
        (
            [name] => Array
                (
                    [0] => 2021_11_30.txt
                    [1] => 2021_12_04.txt
                    [2] => 2021_12_05.txt
                )

            [type] => Array
                (
                    [0] => text/plain
                    [1] => text/plain
                    [2] => text/plain
                )

            [tmp_name] => Array
                (
                    [0] => user_path_to\Temp\phpB7F.tmp
                    [1] => user_path_to\Temp\phpB80.tmp
                    [2] => user_path_to\Temp\phpB90.tmp
                )

            [error] => Array
                (
                    [0] => 0
                    [1] => 0
                    [2] => 0
                )

            [size] => Array
                (
                    [0] => 865
                    [1] => 674
                    [2] => 349
                )
        )
)

Notice how each file uploaded has incremented KEYS for each attribute. Also notice the primary [filename] followed by the secondary {keys] like [name], these can be used to get the record keys and loop through the files using foreach(). We can then use file() to get the file data from ['tmp_name'] using the corresponding [$key] for each file.

foreach($_FILES['filename']['name'] as $key => $filename):

	$lines = file($_FILES['filename']['tmp_name'][$key]);	
	echo "<pre>";
	print_r($lines);
	echo "</pre>";
endforeach;

Again printing it out you might see something like this, with line returns and empty values.

Array
(
    [0] => ----------NEW TRANSACTON-----------

    [1] => TRANSACTION DATE : 09:42 AM, 05 Dec 2021

    [2] => CLIENT ID : C05011

    [3] => CLIENT NAME : SAINT JOE PATRICK

    [4] => AMOUNT : 700,550.00

    [5] => TRANSACTION REF. : 02101

    [6] => 

    [7] => ----------NEW TRANSACTON-----------

    [8] => TRANSACTION DATE : 10:02 AM, 05 Dec 2021

    [9] => CLIENT ID : C40112

    [10] => CLIENT NAME : ABU SALE

    [11] => AMOUNT : 180,650.00

    [12] => TRANSACTION REF. : 02102
)

You can use array_map() and trim off trailing space and line returns,

$newlines = array_map('trim', $lines);

You can then use array_filter() to remove keys => value sets with an empty value.

$newlines = array_filter($newlines, function($value) { return !is_null($value) && $value !== ''; });

The cleaned up version now looks like this.

Array
(
    [0] => ----------NEW TRANSACTON-----------
    [1] => TRANSACTION DATE : 09:42 AM, 05 Dec 2021
    [2] => CLIENT ID : C05011
    [3] => CLIENT NAME : SAINT JOE PATRICK
    [4] => AMOUNT : 700,550.00
    [5] => TRANSACTION REF. : 02101
    [7] => ----------NEW TRANSACTON-----------
    [8] => TRANSACTION DATE : 10:02 AM, 05 Dec 2021
    [9] => CLIENT ID : C40112
    [10] => CLIENT NAME : ABU SALE
    [11] => AMOUNT : 180,650.00
    [12] => TRANSACTION REF. : 02102
)

The data is still not usable… We need a way to define the “headings” that are found in the values. I added them into an array including the extra space between the “heading” and the actual value we want. I also defined a few variables we will use in a bit.

$known_headings = array(
	"TRANSACTION DATE : "
	,"CLIENT ID : "
	,"CLIENT NAME : "
	,"AMOUNT : "
	,"TRANSACTION REF. : "
);

$record = 0;	
$client_list = array();	

Note: These are defined at the top of the page before all foreach loops.

Now we know that each record starts with ----------NEW TRANSACTON----------- and so we can loop though $newlines and if the value (defined as $line) equals that “first record line” we can increment the $record number, which we will use with the heading fields to make the $client_list array.

foreach($newlines as $line):
	if($line == "----------NEW TRANSACTON-----------"){
		$record++;	
	}else{
		
	}
endforeach;

Inside that }else{ section } we can loop through the $known_headings and use strpos() to see if the heading $field is found in the record line. If there is a match we can build the client array with the record number and use str_replace(" : ",'',$field) to clean up the field name and use trim() and str_replace() to remove the $field from the $line and set this as the array value.

foreach($known_headings as $field):
	if(strpos($line, $field) !== false): 
		//Remove headings from the txt line, trim results and set as array value for that field
		$client_list[$record][str_replace(" : ",'',$field)] = trim(str_replace($field, '', $line));
	endif;
endforeach;

All 3 files (11 records) can now be loaded into a single array that would be easy to use for display or inserting into a database.

Array
(
    [1] => Array
        (
            [TRANSACTION DATE] => 08:42 AM, 30 Nov 2021
            [CLIENT ID] => C00001
            [CLIENT NAME] => SALISU BISI
            [AMOUNT] => 16,450.00
            [TRANSACTION REF.] => 00001
        )

    [2] => Array
        (
            [TRANSACTION DATE] => 08:46 AM, 30 Nov 2021
            [CLIENT ID] => C00002
            [CLIENT NAME] => SULE YAYA
            [AMOUNT] => 80,940.00
            [TRANSACTION REF.] => 00002
        )

    [3] => Array
        (
            [TRANSACTION DATE] => 08:51 AM, 30 Nov 2021
            [CLIENT ID] => C00001
            [CLIENT NAME] => SALISU BISI
            [AMOUNT] => 44,900.00
            [TRANSACTION REF.] => 00003
        )

    [4] => Array
        (
            [TRANSACTION DATE] => 08:58 AM, 30 Nov 2021
            [CLIENT ID] => C00003
            [CLIENT NAME] => ADEGOKE BAYO
            [AMOUNT] => 60,000.00
            [TRANSACTION REF.] => 00004
        )

    [5] => Array
        (
            [TRANSACTION DATE] => 09:12 AM, 30 Nov 2021
            [CLIENT ID] => C00004
            [CLIENT NAME] => COKER PRIYE
            [AMOUNT] => 80,200.00
            [TRANSACTION REF.] => 00005
        )

    [10] => Array
        (
            [TRANSACTION DATE] => 09:42 AM, 05 Dec 2021
            [CLIENT ID] => C05011
            [CLIENT NAME] => SAINT JOE PATRICK
            [AMOUNT] => 700,550.00
            [TRANSACTION REF.] => 02101
        )

    [11] => Array
        (
            [TRANSACTION DATE] => 10:02 AM, 05 Dec 2021
            [CLIENT ID] => C40112
            [CLIENT NAME] => ABU SALE
            [AMOUNT] => 180,650.00
            [TRANSACTION REF.] => 02102
        )
)

SO maybe you were hoping for a bit of help as it does take some manipulation to change those files into a usable format.
uploadfile.php (2.1 KB)

Thanks very much, this approach showed me a new way of doing things though I have used, though I have used php file_get_contents() to read three txt files.

Someone’s put transaction data on a machine-accessible URL, including personally identifiable information?

I dont know who you work for, but your security department would like a word or 20 with someone.

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.