Serverless skeleton App with DB data realtime update



Project overview


Jump to: SOLUTION -Project implementation

Problem Statement

Skills & Concepts demonstrated

Insights

The data source is Firebase NoSQL database that triggers event for each update, delete or add action in the DB. Subscribing to event handler and providing custom callback function client has real-time data state.



Problem Statement


The goal of this project is to build reusable skeleton App in MVC architecture that automatically refreshes clients upon any data change in DB.

1-How to achieve separation of business logic, application logic and presentation logic?
2-How to ensure that Model receives realtime data state from the Data source, independent who made the change?
3-How to refresh View with actual data state since the View does not know that Model as entity even exists in the App?
4-How to isolate Model from Data source-specific features and enable easier testing with mock data source?
5-How to separate the concerns of constructing objects and using them, to have loosely coupled program?


Skills & Concepts demonstrated


There are following Java Script and Software Architecture features used in this project

  • Model View Controller
  • Observer
  • Repository
  • Dependency Injection
  • Callback function
  • Array methods forEach, map, find, filter, reduce, every, some
  • Object-oriented programming


  • Insights


    1. MVC architecture ensures isolation of Model (business logic) and View (presentation logic) from the Controller (application logic). Controller calls functions from Model and from the View, subscribes and listens to updating Model and updates View accordingly in the realtime. Model and View are not aware neither of each other nor of the Controller, they simply sit and wait for their functions to be called

    2. When data change happened in the Data Source it triggers event listener in the Observer (Model) and then the Model handles data change and updates its variables with latest values. This data change handling is being executed asynchronously within callback function that Model provided during initialization when subscribed as data change listener

    3. When Model data update happened it triggers event listener in the Observer (Controller) and then the Controller performs View update with latest data state. Although showing latest data state in the View depends on Model update, View and Model are isolated from one another and cannot communicate directly, so the Observer pattern addresses this requirement

    4. The DB-related logic is implemented using Repository pattern. The Model has reference to Repository class which acts as a middle layer that encapsulates database queries, keeping the Model clean and focused on domain logic. Repository class has reference to IDatabase and follows contract defined in it calling its methods. Actual DB handling is implemented in FirestoreDatabase class that extends IDatabase and implements methods that will be called on runtime



    Project implementation


    Back to: RESUME -Project overview

    1 - MVC Application architecture

    2 - Model update on data change

    3 - View update on Model update

    4 - Repository design pattern for proper data persistence handling

    5 - Complete project implementation


    All files this project contains are available on GitHub Repo as 1-MVC-Observer-Repo.zip

    1 - MVC Application architecture


    MVC (Model-View-Controller) is an architectural pattern with primary role to define the architecture of an application by separating concerns—model (data), view (UI), and controller (logic).

  • Model imported DB module
  • View imported no module
  • Controller imports both Model and View
  • App module imports Model, View and Cotroller
  • Constants are defined in separate module and imported where needed
  • Application entry point is app.js where Controller instance is created. Previously created Model and View are passed to Controller constructor.

    // app.js
    import Model from './model.js';
    import View from './view.js';
    import Controller from './controller.js';
    
    // Create instances of Model and View
    const model = new Model();
    const view = new View();
    
    // Inject the instances into Controller
    const controller = new Controller(model, view);

    Controller constructor saves passed Model and View instances and calls init() method on each of them

    // controller.js
    import Model from "./model.js";
    import View from "./view.js";
    
    class Controller {
      constructor(model, view) {
        this.model = model;
        this.view = view;
    
        // Initialize the app
        this.model.init();
        this.view.init();


    2 - Model update on data change



    On initialization Model is subscribed to list for listening changes in the DB, calling subscribeCollection function from dbFirestoreNoSQL module, providing it's method handleChanges as callback function

    // model.js
    class Model {
      init() {
        // Subscribe to the Firestore collection 'Dummy' and listen for changes
        subscribeCollection("Dummy", this.handleChanges.bind(this)); // Pass as callback function
      }
      // Function to handle changes from Firestore and update the Model's data
      handleChanges(newValue) {
        this.data = extractvalues(newValue); // Update the model's data with the new value
      }

    Database handling module - Key points


    1-Function subscribeCollection receives collection and callbackFunction as parameters
    2-It calls onSnapshot function that returns function to be called in order to unsubscribe
    3-Function onSnapshot is called with 2 arguments: the 1st is Firestore collection reference and the 2nd is callback function querySnapshot that runs on data change
    4-Within querySnapshot function is called callbackFunction passed from Model

    // dbFirestoreNoSQL.js
    export function subscribeCollection(collectionName, callbackFunction) {
      if (unsubscribe) // initially null
        unsubscribe();  // Stops listening to the previous item
      collectionListening = collection(db, collectionName);
      // onSnapshot returns a function that can be called to unsubscribe from updates
      unsubscribe = onSnapshot( 
        collectionListening, // the 1st argument: Firestore collection reference to listen to
        (querySnapshot) =>   // the 2nd argument: callback function that runs on data change
        { // querySnapshot callback function begins
          const docsToUpdate = [];
          querySnapshot.forEach((docSnap) => { // calling Firestore's forEach method on QuerySnapshot object
              docsToUpdate.push({ [docSnap.id]: docSnap.data() }); // extracting document data
          });
          callbackFunction(docsToUpdate); // calling Model's callback function with updated data
        } // querySnapshot callback function ends
      ); // onSnapshot call ends
    } // subscribeCollection implementation ends


    Refactoring using named function instead of Arrow function


    READING SEQUENCE TOP-BOTTOM

  • Defined a named function handleSnapshot(querySnapshot), which loops through querySnapshot (which represents all documents in the Firestore collection at this moment)
  • Calls callbackFunction(docsToUpdate), passing the extracted data to the Model.
  • Passed handleSnapshot as the second argument to onSnapshot, so it will be executed whenever Firestore sends data.

  • EXECUTION SEQUENCE

  • (1) onSnapshot is called
  • (2) Firestore immediately retrieves the current state and passes that data as a querySnapshot object to handleSnapshot that is executed right away
  • (3) The data is extracted, stored in docsToUpdate, and sent to Model's callbackFunction
  • // dbFirestoreNoSQL.js
    function handleSnapshot(querySnapshot) {  //(2)
      const docsToUpdate = [];
      querySnapshot.forEach(function(docSnap) { 
          docsToUpdate.push({ [docSnap.id]: docSnap.data() }); 
      });
      callbackFunction(docsToUpdate); //(3)
    }
    // Now, call onSnapshot with this function
    unsubscribe = onSnapshot(collectionListening, handleSnapshot); // (1)
    

    Inside subscribeCollection, callbackFunction is available to all inner functions due to closure. Since handleSnapshot is defined inside subscribeCollection, it inherits access to callbackFunction (Model's method handleChanges) . It is resolved at runtime due to closures — handleSnapshot still has access to callbackFunction, even though it's not explicitly passed in.

    When onSnapshot is called for the first time (Initial Snapshot), Model is subscribed as observer to data changes in the database (Observer Pattern) and provides it's method as callback function (handleChanges):

  • Firestore sets up a listener on the collection, making it an observer
  • Any future changes (add/update/delete) to the collection will trigger the callback function (handleSnapshot) passed as the 2nd argument to onSnapshot call that will prepare data and send it to Model's callback function (handleChanges)

  • 3 - View update on Model update



    After handling changes and updating it's data, Model will notify subscribed listeners about the change. Subscribed listener is Controller, that in ctor adds itself to Model's observer list. Function renderContent for updating view, that Controller calls in ctor, is also called in function onModelUpdate

    // model.js
    class Model {
      constructor() {
          this.observers = [];  // Array to hold observers 
      }
      // Function to handle changes from Firestore and update the model's data
      handleChanges(newValue) {
        this.notifyObservers(); // Notify all observers about the data change
      }
      // Function to add an observer 
      addObserver(observer) {
        this.observers.push(observer);
      }
      // Function to notify all observers that data has changed
      notifyObservers() {
        this.observers.forEach(observer => observer.onModelUpdated(this.data));
      }
    // controller.js
    class Controller {
      constructor(model, view) {
        // Register the Controller as the observer of the Model
        this.model.addObserver(this);  // Controller is the observer now
        // Pass the model data to the View to render
        this.view.renderContent(this.model.data);
      }
      // Function to update the view (called when Model data changes)
      onModelUpdated(data) {
        this.view.renderContent(data); // Pass the updated data to the View
      }


    4 - Repository design pattern for proper data persistence handling



    There are following components and their roles in Repository design pattern

  • Model - has Repository reference and interactcts with it only. Contains attributes and behavior related to the data. It can also include methods for manipulating its own data, but it doesn't directly interact with the database.
  • Repository - handles DB queries. For switch databases (e.g., from SQL Server to MongoDB) changes will made in the repository implementation without modifying the Model. Provides business logic and data management, serving as an intermediary between the Model and the IDatabase.
  • IDatabase - provides contract, unified interface that Repository counts on and calls its methods.
  • FirestoreDatabase - extends IDatabase and implements contract for Firestore or any specific database. Contains the actual implementation of the database operations specific to Firestore.
  • // model.js
    class Model {
      constructor(repository) {
        this.repository = repository;
      }
      init() {
        // Subscribe to the Firestore collection 'Dummy' and listen for changes
        this.repository.subscribeCollection("Dummy", this.handleChanges.bind(this)); 
      }
      // Function to fetch message from Firestore
      async fetchMessage() {
        const message = await this.repository.readDBkey("Dummy", "DummyDoc", "DummyMsg");
        this.data = message;
      }
      // Function to update the data in Firestore
      async updateDataToDB(value) {
        await this.repository.updateDB("Dummy", "DummyDoc", "DummyMsg", value); // Update the value in DB
      }
    }
    
    // app.js
    import Repository from "./repository.js";
    import FirestoreDatabase from "./database/FirestoreDB.js";
    
    // Use Firestore as the database
    const database = new FirestoreDatabase();
    const repository = new Repository(database);
    const model = new Model(repository);


    5 - Complete project implementation



    In the project root there is index.html. Javascript files are placed in js folder, style in css folder and DB handling code in database subfolder of js folder. In the <head> section of index.html the only script file referenced is app.js. It is app entry point, and it imports all other modules. Model and View are isolated from one another an import only common constants they need. All application logic ih handled by Controller that calls functions from Model and View.
    DB handling logic is separated using Repository pattern. All neccessary DB-specific scripts are referenced in FirestoreDB.js. In the model.js the Repository object is passed in constructor, and DB-related methods are called on that object. Repository object calls standard contract-defined methods on IDatabase, which are at runtime resolved to particular methods implemented in FirestoreDatabase class that extends IDatabase interface. In app.js object of FirestoreDatabase is created and passed to created Repository object. It is passed to created Model object which is passed to created Controller object.


    There are all project files:

    <!-- index.html -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
      <meta charset="UTF-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>MVC-Observer-Repository </title>
      <link rel="stylesheet" href="css/style.css">
      <!-- Add Firebase CDN (This includes both Firebase App and Firestore) -->
      <script type="module" src="./js/app.js"></script>
    </head>
    <body>
      <div class = "maindiv">
        <div class="centraldiv">
          <input class = "uicontrols" id = "inputContent"></input> <br />
          <button class = "uicontrols button" id="btnWrite">Write to DB</button>
          <button class = "uicontrols button" id="btnRead">Read from DB</button> <br /><br />
          <label id="output" class = "uicontrols label">Write to DB or Read from DB</label><br />
          <img src = "images/mysql.png" height="50"/>
          <img src = "images/cassandra.png" height="50"/>
          <img src = "images/dynamodb.png" height="50"/>
          <img src = "images/hbase.png" height="50"/>
          <img src = "images/redisdb.png" height="50"/>
          <img src = "images/firebase.png" height="50"/>
          <img src = "images/mongodb.png" height="50"/>
          <img src = "images/postgresql.png" height="50"/>
          <img src = "images/neo4j.png" height="50"/>
          <img src = "images/neptune.png" height="50"/>
        </div>
      </div>
    </body>
    </html>
    /* style.css */
    .uicontrols {
      height: 50px;
      width: 404px;
      font-size: 24px;
      padding: 5px; /* Ensures padding doesn’t add extra height */
      border: 1px solid #ccc; /* Ensures consistency */
      border-radius: 8px;
      box-sizing: border-box; /* Ensures height includes border & padding */
      vertical-align: middle; /* Aligns them properly */
    }
    .button {
      width: 200px;
      margin-top: 5px;
    }
    .label {
      height: 40px;
      margin-top: 5px;
      border-style: none;
      color: #444;
      display:block;  
    }
    .centraldiv {
      position: absolute;
      top: 50%;
      left: 50%;
      transform: translate(-50%, -50%);
    }
    .maindiv {
      position: absolute;
      top: 50%;
      left: 50%;
      transform: translate(-50%, -50%);
      height: 500px;
      width: 800px;
      background-color: #DDF;
      border: 2px solid #888; /* Ensures consistency */
      border-radius: 18px;
    }
    // app.js
    import Model from './model.js';
    import View from './view.js';
    import Controller from './controller.js';
    
    import Repository from "./repository.js";
    import FirestoreDatabase from "./database/FirestoreDB.js";
    
    // Use Firestore as the database
    const database = new FirestoreDatabase();
    const repository = new Repository(database);
    const model = new Model(repository);
    const view = new View();
    // Inject the instances into Controller
    const controller = new Controller(model, view); // App entry point
    // model.js
    import { GAME_LEVELS } from "./constants.js";
    
    class Model {
      constructor(repository) {
        this.data = "";  // Placeholder for storing data
        this.gameLevels = GAME_LEVELS;
        this.repository = repository;
        this.observers = [];  // Array to hold observers (the Controller)
      }
      // Init function to set initial data to "Welcome!"
      init() {
        this.data = "Welcome back!"; // Initialize the data with "Welcome!"
        // Subscribe to the Firestore collection 'Dummy' and listen for changes
        this.repository.subscribeCollection("Dummy", this.handleChanges.bind(this)); // Pass handleChanges as the callback f.
      }
      // Function to handle changes from Firestore and update the model's data
      handleChanges(newValue) {
        const doc = newValue.find(el=>el.hasOwnProperty('DummyDoc'));        
        this.data = doc['DummyDoc'].DummyMsg; // Update the model's data with the new value
        console.log("DB:", this.data); // Log the updated value to the console
        this.notifyObservers(); // Notify all observers (controller) about the data change
      }
      // Function to add an observer (Controller)
      addObserver(observer) {
        this.observers.push(observer);
      }
      // Function to notify all observers that data has changed
      notifyObservers() {
        this.observers.forEach(observer => observer.onModelUpdated(this.data));
      }
      // Function to fetch message from Firestore
      async fetchMessage() {
        const message = await this.repository.readDBkey("Dummy", "DummyDoc", "DummyMsg");
        this.data = message;
      }
      // Function to update the data in Firestore
      async updateDataToDB(value) {
        await this.repository.updateDB("Dummy", "DummyDoc", "DummyMsg", value); // Update the value in DB
      }
    }
    export default Model;
    // view.js
    import { GAME_LEVELS } from "./constants.js";
    class View {
      constructor() {
          this.gameLevels = GAME_LEVELS;
          this.inputElement = document.getElementById("inputContent"); // Reference to input element
      }
      // Function to update the input element with the message
      renderContent(message) {
          this.inputElement.value = message;
      }
      // Init function to set the initial content of inputContent
      init() {
        this.inputElement.value = "";
      }
    }
    export default View;
    // controller.js
    class Controller {
      constructor(model, view) {
        this.model = model;
        this.view = view;
        // Initialize the app (sets initial content in inputContent)
        this.model.init();
        this.view.init();
        // Register the controller as the observer of the model
        this.model.addObserver(this);  // Controller is the observer now
        // Pass the model data to the view to render
        this.view.renderContent(this.model.data);
        // Bind buttons to click event
        document.getElementById("btnRead").addEventListener("click", this.handleReadClick.bind(this));
        document.getElementById("btnWrite").addEventListener("click", this.handleWriteClick.bind(this));    
      }
      // Handle the click event for the Read button
      async handleReadClick() {
        await this.model.fetchMessage(); // Fetch message from Firestore
        this.view.renderContent(this.model.data); // Update the view with the message
      }
      // Handle the click event for the Write button
      async handleWriteClick() {
        const valueToWrite = document.getElementById("inputContent").value; // Get the current value 
        await this.model.updateDataToDB(valueToWrite); // Update the value in Firestore
      }
      // Function to update the view (called when model data changes)
      onModelUpdated(message) {
        this.view.renderContent(message); // Pass the updated message to the view
      }
    }
    export default Controller;    
    // constants.js
    export const GAME_LEVELS = 6;
    // repository.js
    class Repository { 
      constructor(database) {
          this.database = database; // Injected FirestoreDatabase
      }
      async readDBkey(coll, doc, key) {
          return this.database.read(coll, doc, key);
      }
      async updateDB(coll, doc, key, value) {
          return this.database.update(coll, doc, key, value);
      }
      subscribeCollection(coll, callback) {
        this.database.subscribeCollection(coll, callback);
      }
    }
    export default Repository;
    // database/IDatabase.js
    class IDatabase {
      async read(coll, doc, key) {
          throw new Error("read() must be implemented");
      }
      async update(coll, doc, key, value) {
          throw new Error("update() must be implemented");
      }
      subscribeCollection(coll, callback) {
        throw new Error("subscribeCollection() must be implemented");
      }
    }
    export default IDatabase;
    // database/FirestoreDatabase.js
    import IDatabase from "./IDatabase.js";
    import { initializeApp } from "https://www.gstatic.com/firebasejs/10.11.1/firebase-app.js";
    import { getFirestore, collection, getDoc, updateDoc, onSnapshot, doc } 
      from "https://www.gstatic.com/firebasejs/10.11.1/firebase-firestore.js";
    
    // Firebase config
    const firebaseConfig = {
      apiKey: "AIzaSyCBSwWx50dibvQQcGd2-ZtWiAghCi1EMI0",
      authDomain: "mastermindbv75.firebaseapp.com",
      projectId: "mastermindbv75",
      storageBucket: "mastermindbv75.firebasestorage.app",
      messagingSenderId: "245901977986",
      appId: "1:245901977986:web:d7716512d026ff10fc479b"
    };
    
    class FirestoreDatabase extends IDatabase {
        constructor() {
            super();
            // Initialize Firebase inside the class
            this.app = initializeApp(firebaseConfig);
            this.db = getFirestore(this.app); // Firestore instance
            this.unsubscribe = null; // Store unsubscribe function
        }
        async read(coll, docId, key) {
            const docRef = doc(this.db, coll, docId);
            const snapshot = await getDoc(docRef);
            return snapshot.exists() ? snapshot.data()[key] : null;
        }
        async update(coll, docId, key, value) {
            const docRef = doc(this.db, coll, docId);
            await updateDoc(docRef, { [key]: value });
            return { key, value };
        }
        subscribeCollection(coll, callback) {
          // First, unsubscribe from the previous listener (if any)
          if (this.unsubscribe) 
            this.unsubscribe();
          const collectionRef = collection(this.db, coll);
          this.unsubscribe = onSnapshot(collectionRef, (querySnapshot) => {
              const docsToUpdate = [];
              querySnapshot.forEach((docSnap) => {
                  docsToUpdate.push({ [docSnap.id]: docSnap.data() });
              });
              callback(docsToUpdate);
          });
        }
    }
    export default FirestoreDatabase;

    About the author

    Barry The Analyst has been working from May 2022 untill July 2023 with one big company in Ireland, and since July 2023 has worked with one big company in Germany. This blog was started as a private note collection in order to (re)develop, refresh and strengthen essential skills of Data Analysis field. The content was generated on the way of learning path, with a degree in Electrical Engineering and experience in C++/SQL. Private notes turned into public blog to share the knowledge and provide support to anyone who would find this interesting...

     

    Who is this for?

    This is not yet another basic course that would explain elementary concepts. There are many basic features of tools and technologies that are skipped in blog articles and portfolio projects here presented. If you are not an absolute beginner in SQL/databases, Excel, Python and Power BI this might be useful material if you would like to advance a career as Data Analyst and/or Python Developer ...

    This template downloaded form free website templates