Hello and welcome! Today, we're exploring practical data manipulation techniques in JavaScript. We'll use JavaScript arrays to represent our data stream and perform projection, filtering, and aggregation. And here's the star of the show: our operations will be neatly packaged within a JavaScript class! No mess, all clean code.
Data manipulation is akin to being a sculptor but for data. We chisel and shape our data to get the desired structure. JavaScript arrays
are perfect for this, and our operations will be conveniently bundled inside a JavaScript class
. So, let's get our toolbox ready! Here's a simple JavaScript class, DataStream
, that will serve as our toolbox:
JavaScript1class DataStream { 2 constructor(data) { 3 this.data = data; 4 } 5}
Our first stop is data projection. Think of it like capturing a photo of our desired features. Suppose we have data about people. If we're only interested in names and ages, we project our data to include just these details. We'll extend our DataStream
class with a projectData
method for this:
JavaScript1class DataStream { 2 constructor(data) { 3 this.data = data; 4 } 5 6 projectData(keys) { 7 const projectedData = this.data.map(d => { 8 let newObj = {}; 9 keys.forEach(key => { 10 newObj[key] = d[key] !== undefined ? d[key] : null; 11 }); 12 return newObj; 13 }); 14 return projectedData; 15 } 16} 17 18// Let's use it! 19const ds = new DataStream([ 20 { 'name': 'Alice', 'age': 25, 'profession': 'Engineer' }, 21 { 'name': 'Bob', 'age': 30, 'profession': 'Doctor' }, 22]); 23 24const projectedDs = ds.projectData(['name', 'age']); 25console.log(projectedDs); // Outputs: [{ 'name': 'Alice', 'age': 25 }, { 'name': 'Bob', 'age': 30 }]
As you can see, we now have a new array with just the names and ages!
Next, we have data filtering, which is like cherry-picking our preferred data entries. We'll extend our DataStream
class with a filterData
method that uses a "test" function to filter data:
JavaScript1class DataStream { 2 constructor(data) { 3 this.data = data; 4 } 5 6 // ... other methods ... 7 8 filterData(testFunc) { 9 const filteredData = this.data.filter(testFunc); 10 return filteredData; 11 } 12} 13 14// Applying it: 15const ds = new DataStream([ 16 { 'name': 'Alice', 'age': 25, 'profession': 'Engineer' }, 17 { 'name': 'Bob', 'age': 30, 'profession': 'Doctor' }, 18]); 19 20const ageTest = x => x['age'] > 26; // our "test" function 21const filteredDs = ds.filterData(ageTest); 22console.log(filteredDs); // Outputs: [{ 'name': 'Bob', 'age': 30, 'profession': 'Doctor' }]
With the filter method, our output is an array with only Bob’s data, as he's the only one who passes the 'age over 26' test.
Last is data aggregation, where we condense our data into a summary. We will add an aggregateData
method to our DataStream
class for this:
JavaScript1class DataStream { 2 constructor(data) { 3 this.data = data; 4 } 5 6 // ... other methods ... 7 8 aggregateData(key, aggFunc) { 9 const values = this.data.map(d => d[key]); 10 return aggFunc(values); 11 } 12} 13 14// Let's put it to use 15const ds = new DataStream([ 16 { 'name': 'Alice', 'age': 25, 'profession': 'Engineer' }, 17 { 'name': 'Bob', 'age': 30, 'profession': 'Doctor' }, 18]); 19 20const averageAge = ds.aggregateData('age', ages => ages.reduce((a, b) => a + b) / ages.length); 21console.log(averageAge); // Outputs: 27.5
With this script, we get the average age of Alice and Bob, which is 27.5
.
Now, let's combine projection, filtering, and aggregation to see the collective power of these techniques. We'll extend our example to demonstrate this flow:
- Data Projection: Choose only the desired fields.
- Data Filtering: Filter the data based on certain conditions.
- Data Aggregation: Summarize the filtered data.
We'll modify our DataStream
class to include all the methods and then use them together in a workflow. On top of that, projection and filtering methods will now return an instance of DataStream
, not an array as before, so that we can chain these methods when calling them:
JavaScript1class DataStream { 2 constructor(data) { 3 this.data = data; 4 } 5 6 projectData(keys) { 7 const projectedData = this.data.map(d => { 8 let newObj = {}; 9 keys.forEach(key => { 10 newObj[key] = d[key] !== undefined ? d[key] : null; 11 }); 12 return newObj; 13 }); 14 return new DataStream(projectedData); // Return a new DataStream object for chaining 15 } 16 17 filterData(testFunc) { 18 const filteredData = this.data.filter(testFunc); 19 return new DataStream(filteredData); // Return a new DataStream object for chaining 20 } 21 22 aggregateData(key, aggFunc) { 23 const values = this.data.map(d => d[key]); 24 return aggFunc(values); 25 } 26} 27 28// Example usage 29const ds = new DataStream([ 30 { 'name': 'Alice', 'age': 25, 'profession': 'Engineer', 'salary': 70000 }, 31 { 'name': 'Bob', 'age': 30, 'profession': 'Doctor', 'salary': 120000 }, 32 { 'name': 'Carol', 'age': 35, 'profession': 'Artist', 'salary': 50000 }, 33 { 'name': 'David', 'age': 40, 'profession': 'Engineer', 'salary': 90000 }, 34]); 35 36// Step 1: Project the data to include only 'name', 'age', and 'salary' 37const projectedDs = ds.projectData(['name', 'age', 'salary']); 38 39// Step 2: Filter the projected data to include only those with age > 30 40const filteredDs = projectedDs.filterData(x => x['age'] > 30); 41 42// Step 3: Aggregate the filtered data to compute the average salary 43const averageSalary = filteredDs.aggregateData('salary', salaries => salaries.reduce((a, b) => a + b) / salaries.length); 44console.log(averageSalary); // Outputs: 70000.0
Here:
- Projection: We choose only the
name
,age
, andsalary
fields from our data. TheprojectData
method now returns aDataStream
object, allowing us to chain multiple operations. - Filtering: We filter the projected data to include only those persons whose age is greater than 30. The
filterData
method also returns aDataStream
object for chaining. - Aggregation: We calculate the average salary of the filtered data. The final output shows the average salary for those aged over 30, which is
70,000
.
By combining these methods, our data manipulation becomes both powerful and concise. Try experimenting and see what you can create!
Brilliant job! You've now grasped the basics of data projection, filtering, and aggregation on JavaScript arrays. Plus, you've learned to package these operations in a JavaScript class
— a neat bundle of reusable code magic!
Now, why not try applying these fresh skills with some practice exercises? They're just around the corner. Ready? Let's dive into more fun with data manipulation!