Prerequisites

Before using MooseJS, ensure you have the following dependencies installed on your machine:

To check if these dependencies are installed, run the following command in your terminal:

git --version && node --version && docker --version

Create Moose App

In your terminal, run:

npx create-moose-app@latest my-moose-app

This will automatically create a new project folder and initialize a skeleton Moose app with the entire project structure you need to get started.

Install Dependencies

Navigate to your project directory and install project dependencies:

cd my-moose-app && npm install

Start Local Dev Server

Start your local development server to run your entire app,

npx moose-cli dev

This server operates all the infrastructure powering your application: a web server with API endpoints to ingest & consume data, a Redpanda platform to buffer data, and a Clickhouse database to store and query data for insights. The local dev server configures these components based on your code and updates them in real time as your application code changes.

Inspect Infrastructure

Check that the infrastructure is setup according to the default starter code in your project by running:

npx moose-cli ls

You should see the following table in your terminal:

Open Project in IDE

VSCode Users: Install Recommended Extensions

code .

Opening your project in VSCode will prompt a recommendation to install the SQLTools (opens in a new tab) and SQLTools ClickHouse Driver (opens in a new tab) extensions. Installing these allows Moose to auto-configure them for your local Clickhouse database, letting you explore and query your data immediately.

Explore Data Models

In your IDE file explorer, navigate to the /app folder of your project and open the datamodels directory. You should see a file named models.ts . Open the file and inspect the data models:

import { Key } from "@514labs/moose-lib";
 
export interface UserActivity {
  eventId: Key<string>;
  timestamp: string;
  userId: string;
  activity: string;
}
 
export interface ParsedActivity {
  eventId: Key<string>;
  timestamp: Date;
  userId: string;
  activity: string;
}

The file contains schema definitions for two data models: UserActivity and ParsedActivity. Each model defines the structure of the data that will be ingested into the Clickhouse database. Recall from the moose-ls output that Moose has automatically created ingestion points and tables in your database for these data models.

Ingest Data

Let's send some sample data to the UserActivity data model via the /ingest/UserActivity/0.0 endpoint that Moose created for us. Run the following command:

curl -X POST \
-H "Content-Type: application/json" \
-d '{"eventId": "1234567890", "timestamp": "2019-01-01 00:00:01", "userId": "123456", "activity": "click"}' \
http://localhost:4000/ingest/UserActivity
 

You should see a bunch of messages in the CLI confirming that Moose received the data. Let's double-check that the new data landed in our UserActivity_0_0 table in our database.

In your SQL explorer, execute the following query:

SELECT * FROM local.UserActivity

Query Data

Follow the steps below to get started with querying your data using either the SQLTools VSCode extension or your preferred database explorer tool.

Execute the following query:

SELECT * FROM local.UserActivity

You should see the data you ingested in the UserActivity table.

Next Steps

Congrats! You've successfully set up a Moose project, ingested data, and confirmed that it landed in your database. You're now ready to dive deeper and explore all the exciting capabilities that Moose has to offer. Here are some suggested next steps:

Learn Other Moose Primitives

After ingesting the data, you might have noticed the CLI printed the following message: Received UserActivity_0_0 -> ParsedActivity_0_0 1 message(s). This indicates that Moose successfully transformed the data from the UserActivity model to the ParsedActivity model. Moose did this by executing the transformation function defined in the UserActivity__ParsedActivity.ts file in the /functions/ folder on the UserActivity data sample you ingested. You can learn more about how to define execute custom transformation functions on your data by checking out the Streaming Functions documentation.

Streaming functions are one of the three remaining Moose primitives left for you to explore. For a quick overview on how to get started with Streaming Functions and the other primitives, refer to the Quick Reference section.

Add a New Data Model

Define and save a new data model in the /datamodels/models.ts file to automatically create ingestion points and tables in your database for the new model. Check out the Data Models docs to learn about other ways to take advantage of Moose's data modeling capabilities, including the supported types you can leverage in your schema definitions and how to get started with Moose's auto-generated ingest SDKs to start capturing data from your other applications.

Understand your Project Structure

Explore the docs to learn more about your project structure or the architecture of your new app.