How to safely handle Mongoose transactions inside Next.js 15 Server Actions?

To safely handle Mongoose transactions within Next.js 15 Server Actions and prevent orphaned data, you need to use Mongoose sessions and transactions. The errors you're encountering, such as "custom sessions" or "unhandled promise rejections on concurrent connection states," often stem from not correctly initiating, using, and ending sessions, or from your MongoDB instance not being configured as a replica set (a strict requirement for transactions).

Here's how to correctly implement Mongoose transactions in your createQuestion Server Action:

Key Requirements for Mongoose Transactions

MongoDB Replica Set: This is the most crucial requirement. Transactions only work if your MongoDB instance is running as a replica set. If you're using a standalone MongoDB instance, transactions will not function. For local development, you can set up a single-node replica set.
Mongoose Sessions: Transactions are tied to a Mongoose session. You must explicitly start a session, use it for all operations within the transaction, and then end it.
session Option: Every Mongoose operation (like create, findByIdAndUpdate, save, updateMany, etc.) that is part of a transaction must explicitly pass the { session } option.
try...catch...finally: Robust error handling is essential. If any operation within the transaction fails, you must abort the transaction. The session must always be ended.

Updated Server Action Structure

First, ensure your dbConnect utility is robust and only establishes a connection if one doesn't already exist. Then, modify your Server Action as follows:

1// lib/actions/question.action.ts
2"use server";
3
4import dbConnect from "@/lib/mongoose";
5import Question from "@/database/question.model";
6import User from "@/database/user.model";
7import mongoose from "mongoose"; // Import mongoose to access startSession
8
9export async function createQuestion(params: any) {
10  await dbConnect(); // Ensure connection is established
11
12  const { title, content, authorId } = params;
13
14  // 1. Start a Mongoose session
15  const session = await mongoose.startSession();
16
17  try {
18    // 2. Start the transaction
19    session.startTransaction();
20
21    // 3. Create a new document in the Question collection
22    // Note: When using .create() with a session, Mongoose expects an array of documents.
23    const [newQuestion] = await Question.create(
24      [{ title, content, author: authorId }],
25      { session }
26    );
27
28    if (!newQuestion) {
29      throw new Error("Failed to create question.");
30    }
31
32    // 4. Increment the reputation field in the User collection
33    const updatedUser = await User.findByIdAndUpdate(
34      authorId,
35      { $inc: { reputation: 5 } },
36      { session, new: true } // { new: true } returns the updated document
37    );
38
39    if (!updatedUser) {
40      throw new Error("User not found or reputation update failed.");
41    }
42
43    // 5. Commit the transaction if all operations succeed
44    await session.commitTransaction();
45
46    // Return the created question, parsed for safe transfer over network
47    return JSON.parse(JSON.stringify(newQuestion));
48
49  } catch (error) {
50    // 6. If any error occurs, abort the transaction
51    await session.abortTransaction();
52    console.error("Transaction failed:", error);
53    // Re-throw the error so the caller knows something went wrong
54    throw error;
55  } finally {
56    // 7. Always end the session, regardless of success or failure
57    session.endSession();
58  }
59}

1// lib/actions/question.action.ts
2"use server";
3
4import dbConnect from "@/lib/mongoose";
5import Question from "@/database/question.model";
6import User from "@/database/user.model";
7import mongoose from "mongoose"; // Import mongoose to access startSession
8
9export async function createQuestion(params: any) {
10  await dbConnect(); // Ensure connection is established
11
12  const { title, content, authorId } = params;
13
14  // 1. Start a Mongoose session
15  const session = await mongoose.startSession();
16
17  try {
18    // 2. Start the transaction
19    session.startTransaction();
20
21    // 3. Create a new document in the Question collection
22    // Note: When using .create() with a session, Mongoose expects an array of documents.
23    const [newQuestion] = await Question.create(
24      [{ title, content, author: authorId }],
25      { session }
26    );
27
28    if (!newQuestion) {
29      throw new Error("Failed to create question.");
30    }
31
32    // 4. Increment the reputation field in the User collection
33    const updatedUser = await User.findByIdAndUpdate(
34      authorId,
35      { $inc: { reputation: 5 } },
36      { session, new: true } // { new: true } returns the updated document
37    );
38
39    if (!updatedUser) {
40      throw new Error("User not found or reputation update failed.");
41    }
42
43    // 5. Commit the transaction if all operations succeed
44    await session.commitTransaction();
45
46    // Return the created question, parsed for safe transfer over network
47    return JSON.parse(JSON.stringify(newQuestion));
48
49  } catch (error) {
50    // 6. If any error occurs, abort the transaction
51    await session.abortTransaction();
52    console.error("Transaction failed:", error);
53    // Re-throw the error so the caller knows something went wrong
54    throw error;
55  } finally {
56    // 7. Always end the session, regardless of success or failure
57    session.endSession();
58  }
59}

Explanation of Changes:

import mongoose from "mongoose";: You need to import the mongoose object itself to access mongoose.startSession().
await dbConnect();: This ensures that a database connection is established or reused before any database operations begin.
const session = await mongoose.startSession();: Initiates a new Mongoose session. This session will be used for all operations within the transaction.
session.startTransaction();: Marks the beginning of your transaction. All operations after this point are part of the transaction.
Question.create([{ ... }], { session }):
- create() method, when used with a session, expects an array of documents, even if you're only creating one.
- The { session } option is crucial for associating the operation with the current transaction.
- We use array destructuring [newQuestion] to get the single created document from the array returned by create().
User.findByIdAndUpdate(authorId, { ... }, { session, new: true }):
- Again, the { session } option links this update to the active transaction.
- { new: true } ensures that the updatedUser variable holds the modified user document after the update.
await session.commitTransaction();: If all operations inside the try block complete without errors, this command makes all changes permanent in the database.
catch (error) block: If any operation fails, the catch block is executed.
- await session.abortTransaction();: This is critical. It rolls back all changes made during the transaction, ensuring data consistency (e.g., if user reputation fails, the question creation is undone).
- console.error("Transaction failed:", error); for debugging.
- throw error; re-throws the error so your client-side code can handle it (e.g., display an error message).
finally block:
- session.endSession();: This is equally critical. It releases the session resources. This must always happen, regardless of whether the transaction succeeded or failed, to prevent connection leaks and ensure future transactions can be opened.

By following this pattern, you can safely handle Mongoose transactions within your Next.js Server Actions, ensuring atomicity for your database operations.