This example demonstrates how to import external files using the TAR functionality and how to use built-in libraries like NumPy. In this example, the user specifies an input file (defined as an input parameter) that should already be packaged in the fs.tar file. The script will import the NumPy library, open the file, perform some basic operations on the data, and then display an output value.
Click here to see the full list of compatible libraries
Using Truebit to Execute NumPy example
We use Truebit to execute the NumPy task to get a verified result.
The code
You can find the Numpy example within the folder truebit-nextgen-examples/function-tasks/js/numpy-pandas
📄 task.py
📄 create-fs.sh
📄 gendata.py
task.py
task.py
import numpy as np
import os
def main():
# Load filepath of input CSV
try:
with open("input.txt", "r") as file:
input_file = file.read().strip()
except FileNotFoundError:
print("Error: The file 'input.txt' does not exist.")
return
except Exception as e:
print(f"An error occurred while reading 'input.txt': {e}")
return
# Read the input file (assuming it contains CSV data)
data = []
try:
with open(input_file, 'r') as file:
header = file.readline().strip().split(',')
for line in file:
values = line.strip().split(',')
data.append(values)
except FileNotFoundError:
print("Error: The file '{input_file}' does not exist.")
return
except Exception as e:
print(f"An error occurred while reading '{input_file}': {e}")
return
# Convert columns to appropriate types
A = np.array([float(row[0]) for row in data])
B = np.array([float(row[1]) for row in data])
C = np.array([row[2] for row in data])
# Display the original data
original_output = "Original Data:\n" + str(data)
# Calculate basic statistics using numpy
mean_A = np.mean(A)
std_A = np.std(A)
sum_B = np.sum(B)
max_B = np.max(B)
statistics = (
f"\nMean of column 'A': {mean_A}\n"
f"Standard deviation of column 'A': {std_A}\n"
f"Sum of column 'B': {sum_B}\n"
f"Maximum value of column 'B': {max_B}"
)
# Add a new column with the natural logarithm of 'B' values using numpy
log_B = np.log(B + 1) # +1 to avoid log(0)
# Create a correlation matrix using numpy
correlation_matrix = np.corrcoef(A, B)
correlation_output = "Correlation matrix between 'A' and 'B':\n" + str(correlation_matrix)
# Normalize column 'A' using numpy
A_normalized = (A - mean_A) / std_A
# Filter rows where column 'A' is greater than its mean
filtered_data = np.array([row for row in data if float(row[0]) > mean_A])
# Add a date column (as a string for simplicity)
dates = np.arange('2023-01-01', '2024-01-01', dtype='datetime64[D]').astype(str)
# Group by column 'C' (assuming C is categorical with values 'X', 'Y', 'Z')
unique_C, indices_C = np.unique(C, return_inverse=True)
group_means_A = [np.mean(A[indices_C == i]) for i in range(len(unique_C))]
grouped_output = "Mean of 'A' grouped by 'C':\n" + str(dict(zip(unique_C, group_means_A)))
# Apply a lambda function to create a new column 'A_category'
A_category = np.array(['High' if x > 50 else 'Low' for x in A])
# Handling missing values: introduce some NaN values in 'B' and then fill them
B_with_nan = B.copy()
B_with_nan[::3] = np.nan
B_filled = np.where(np.isnan(B_with_nan), np.nanmean(B_with_nan), B_with_nan)
# Prepare the output to be written to output.txt
output_content = (
original_output + statistics + "\n\n" +
"Log of B:\n" + str(log_B) + "\n\n" +
"Normalized A:\n" + str(A_normalized) + "\n\n" +
"Filtered Data (A > Mean of A):\n" + str(filtered_data) + "\n\n" +
correlation_output + "\n\n" +
grouped_output + "\n\n" +
"A Category:\n" + str(A_category) + "\n\n" +
"B with NaNs filled:\n" + str(B_filled)
)
# Save the results to output.txt
with open('output.txt', 'w') as file:
file.write(output_content)
print(f"Results written to 'output.txt'.")
if __name__ == "__main__":
main()
In/Out parameters
In order to send parameters to the Truebit task, you need to open the "input.txt" file and get the value from there. For now, only one input parameter is allowed.
with open("input.txt", "r") as file:
input_string = file.read().strip()
In order to retrieve the output value from the Truebit task, you need save it in the "output.txt" file.
with open("output.txt", "w") as file:
file.write(output_string)
create-fs.sh
This example uses external files to import data for processing. The create-fs.sh script is a batch file that runs the tar command to package the files into a single archive. The resulting fs.tar file will be used as input data in the Function Task.
Try out
Step 1: Create the NumPy example source code
Within the numpy folder you will find a file called task.py.
Step 2: Build the source code
Execute the build command against Truebit node to get an instrumented Truebit task
Building the function task
Your function task is ready.
Use the following TaskId for execution or deployment:
py_0135863f3dc093c2c2286fd8584c5d1313fa14480e6de7efa54ca4214647842f/1.0.0
The taskId will always starts with the language prefix + "_" + cID
Executing the function Task
Execution Id: 5f86c995-82a2-467d-aec0-da668ae96b32
Task executed with status: succeed
Task executed with result: Original DataFrame:
A B C
17 0.985450 Y
31 0.199326 Y
18 0.432207 Z
69 0.609930 Z
8 0.193569 Y
58 0.113935 Y
54 0.656381 Y
63 0.269771 Z
91 0.790736 X
35 0.999061 Z
Mean of column 'A': 44.4
Standard deviation of column 'A': 25.377943179067916
Sum of column 'B': 5.250366498443773
Maximum value of column 'B': 0.9990610723341307
Modified DataFrame:
A B C log_B A_normalized date A_category B_filled
17 NaN Y 0.685846 -1.079678 2024-01-01 Low 0.333257
31 0.199326 Y 0.181760 -0.528018 2024-01-02 Low 0.199326
18 0.432207 Z 0.359217 -1.040273 2024-01-03 Low 0.432207
69 NaN Z 0.476191 0.969346 2024-01-04 High 0.333257
8 0.193569 Y 0.176948 -1.434316 2024-01-05 Low 0.193569
58 0.113935 Y 0.107899 0.535898 2024-01-06 High 0.113935
54 NaN Y 0.504635 0.378281 2024-01-07 High 0.333257
63 0.269771 Z 0.238837 0.732920 2024-01-08 High 0.269771
91 0.790736 X 0.582627 1.836240 2024-01-09 High 0.790736
35 NaN Z 0.692678 -0.370400 2024-01-10 Low 0.333257
Filtered DataFrame (A > Mean of A):
A B C log_B A_normalized
69 0.609930 Z 0.476191 0.969346
58 0.113935 Y 0.107899 0.535898
54 0.656381 Y 0.504635 0.378281
63 0.269771 Z 0.238837 0.732920
91 0.790736 X 0.582627 1.836240
Correlation matrix between 'A' and 'B':
[[1. 0.09016586]
[0.09016586 1. ]]
Mean of 'A' grouped by 'C':
C A
X 91.00
Y 33.60
Z 46.25
Task resource usage:
┌───────────────┬───────────────┬───────────────┬───────┬─────────────┬──────────┐
│ (index) │ Unit │ Limits │ Peak │ Last status │ Capacity │
├───────────────┼───────────────┼───────────────┼───────┼─────────────┼──────────┤
│ Gas │ 'Steps' │ 1099511627776 │ 'N/A' │ 15073880553 │ 73 │
│ Nested calls │ 'Calls' │ 65536 │ 400 │ 0 │ 164 │
│ Frame memory │ 'Bytes' │ 524288 │ 5054 │ 0 │ 104 │
│ System memory │ '64 Kb pages' │ 4096 │ 2232 │ '𝚫 1' │ 2 │
└───────────────┴───────────────┴───────────────┴───────┴─────────────┴──────────┘
We will use the to and our source code. Once the code is finished, we will it to the coordination hub so that everyone who knows the namespace and taskname.
Execute the command against the Truebit node to test our Algorithm. You will need to submit the instrumented task id + the input parameter.
Last, but not least, Execute the command to deploy our task to the coordination hub, so that anyone with the namespace, taskname and the can execute it.
