Circular imports and how to prevent them

Circular imports and how to prevent them

In this post, circular imports are the focus. What they are, why they are bad and some ideas and techniques for fixing and preventing them.

What are circular imports?

A circular import, also called circular dependency or import cycle, occurs when two modules depend on each other either directly or indirectly through a chain of imported modules. Let’s take a look at a simple example:

import { B } from 'b.ts'

console.log(`${B}`)

export const A = 'symbol A'

import { A } from 'a.ts'

console.log(`${A}`)

export const B = 'symbol B'

Why are they bad?

Circular imports are a code smell that shows that an application modules form a dependency graph and not a tree. This makes it hard to reason about specific modules in isolation as you may end up in a situation that many subsystems depends on each other. It also makes it hard to evolve the code and separate some modules into either external libraries, micro services or just more decoupled modules that different teams can own and maintain independently.

But beyond this theoretical and long term problems, they also expose short term challenges and bugs. DragonFly runs in multiple different Javascript execution environments:

  • On mobile, the code is pre-compiled into bytecode and executed by the Hermes Javascript interpreter.

  • On web, the code is bundled and transpiled with Webpack and run by different browsers such as Chrome, FireFox or Safari.

  • On test runs, the code is executed by Jest, which runs in a Node.js environment with the V8 Javascript interpreter.

Each of these environments handles circular imports in a different way as this behavior is not defined in the Javascript specification. Based in our experience, the Metro bundler and the Hermes interpreter seems to be ok with circular imports on the mobile environment and they haven’t created bugs so far.

However, both on web and test execution environments we have had significant challenges in the past and the present. Not all circular imports break the web but some of them do and when the browser loads the DragonFly bundle, a circular import can cause an error and the whole bundle fails to load. This means any RN experience will fail upon some circular imports. These errors manifest as undefined reference exceptions and it’s not easy to associate them with a circular import problem.. Unfortunately, we haven’t been able to detect what circular imports breaks the web and which ones don’t. So a deeper dive may be required to learn more.

On the test execution environment, Node.js algorithm to handle circular imports can cause tests to fail. In order to break an infinite loop during an import cycle, Node.js will stop processing module a.ts (from the above example) and assign undefined to symbol A so it can finish processing module b.ts. This means that when module b.ts tries to use the symbol A it will fail as A is undefined. For more complicated import cycles that spawn multiple steps in the import chain, changing the import order sometimes fixes the problem as depending on when some symbols are exported and used you may find a combination that avoid the undefined behavior. Needless to say that this creates extremely fragile code. One modification in a module can cause some totally unrelated tests to fail and this is quite tricky to debug.

Are there circular imports in the DragonFly codebase?

In the original DragonFly application tech design, there is a fairly defined architecture with several modules and their relationships. The following diagram is a simplified version of that architecture for the purposes of this discussion.

Fig 1

The arrows represent valid import directions. Meaning that the router module can import pages. Pages can import Bauhaus components, partials, capabilities, GQL client, stores and utilities. However, pages are not supposed to import native modules. And more importantly, the reverse direction should not happen: Bauhaus components should not import DragonFly pages, stores should not import partials, utilities should not import capabilities and so on.

The reality is that in DragonFly there are many imports in the wrong direction and they cause many import cycles. We recently went from over 1400 cycles to about 500 and we were able to reduce it under 100 just by ignoring some modules in this cycle analysis.

Some examples of these imports that violate this architecture diagram are:

  • PlayQueue and Library native modules import the Playback store.

  • Library native module imports the bagOfTracks (Maestro) and podcastEpisodeDetail pages

  • NowPlaying native module imports the nonBlockingUpsellReason capability

  • VerticalItem partial imports the Browse Home page

  • HorizontalItem partial imports the artistDetails, lifeEvent and search pages

  • The gql2sql module (GQL client) imports several pages such as seeMoreRecommendations, bagOfTracks, artistDetail, carModeHome, playlistDetail and many more.

The majority of import cycles in DragonFly are not as simple as the example mentioned at the beginning of this article. Instead, these cycles involve an import chain of 3 or more steps. So they are not easy to detect, specially during code reviews. For this reason, we have recently introduced a command that gets executed as part of the CI, that will automatically detect circular imports. You can run it locally with yarn circular.

Here is an example of non trivial cycle on the current main branch:

components/index.ts
> components/MenuContainer/MenuContainer.tsx
 > partials/modals/OverflowMenu/OverflowMenu.tsx
  > partials/modals/OverflowMenu/menus/album/AlbumMenu.tsx
   > partials/actionComponents/Share/Share.tsx
    > partials/actionHooks/share/useShareAction.tsx
     > partials/actionHooks/share/fragmentMappers/index.ts
      > partials/actionHooks/share/fragmentMappers/albumToShare.ts
       > utils/share/shareUtils.ts
        > pages/reelsRecap/common/data/ReelStoryData.ts
         > pages/reelsRecap/year-in-review/YIRReelsExperience.ts
          > pages/reelsRecap/common/components/utils/ReelConfigProcessor.tsx
           > pages/reelsRecap/common/components/utils/FrameValidator.tsx
            > components/index.ts

As you can see, the main problem is that utils/share/shareUtils.ts is importing elements from the page reelsRecap.

How to remove circular imports?

There are 3 main techniques to remove circular imports:

  • Separating type definitions from type implementations

  • Removing index.ts modules that export everything under them.

  • Refactoring components to make them more decoupled

The first two techniques are simple and straighforward to implement but unfortunately they only fix the low hanging fruit. The most complicated circular imports usually require bigger refactor that involves more decoupling between separate modules in the application.

Let’s talk about the first two techniques.

Separating type definitions from type implementations

This technique is quite old and predates many programming languages. In C and C++, it’s common to write type definitions and function signatures in separate .h files while the implementation of these types and functions is written in .c/.cpp files.

In TypeScript the same can be achieved by moving just the types declaration, interfaces, enums and other type constructs into separate types.ts files. Many times a circular import can be resolved this way as one module just needs to import types from the other module. Moving the types to a third file usually allows this new module to not require any import and this way it breaks the cycle.

Code before:

import { type PlayMode } from 'upsells.ts'

export type Tier = 'Free' | 'Prime' | 'AMU'

export getPlayMode = (tier: Tier): PlayMode => {
  if (tier === 'Free' || tier === 'Prime') {
      return 'Shuffle'
  } else {
      return 'OnDemand'
  }
}

import { type Tier } from 'getPlayMode.ts'

export type PlayMode = 'OnDemand' | 'Shuffle'

export shouldShowUpsell = (tier: Tier, requestedPlayMode: PlayMode) => boolean {
  return (tier === 'Free' || tier === 'Prime') && requestedPlayMode == 'OnDemand'
}

Code after:

export type Tier = 'Free' | 'Prime' | 'AMU'
export type PlayMode = 'OnDemand' | 'Shuffle'

import { type Tier, type PlayMode } from 'types.ts'

export getPlayMode = (tier: Tier): PlayMode => {
  if (tier === 'Free' || tier === 'Prime') {
      return 'Shuffle'
  } else {
      return 'OnDemand'
  }
}

import { type Tier, type PlayMode } from 'types.ts'

export shouldShowUpsell = (tier: Tier, requestedPlayMode: PlayMode) => boolean {
  return (tier === 'Free' || tier === 'Prime') && requestedPlayMode == 'OnDemand'
}

Fig 2

One important thing to remember is that types.ts files should only ever import other types definitions files and never files with implementation code.

Removing index.ts files that just re-export everything under them

A common pattern to simplify imports is to have an index.ts file that just re-export all interesting symbols exported by its submodules. Something like this:

// src/services/foobar/index.ts

export * from './foo.ts'
export * from './bar.ts'

Now if another module wants to import something from foo, they will do this:

// another module

import { Foo } from '~/services/foobar'

However, they are also invoking the import from the bar.ts module just because the index.ts file also imports it. If the foobar subsystem does not have dependencies on anything else, that’s probably fine. But if bar.ts is importing things that comes outside foobar then our module is now extending its import chain without even knowing it.

In DragonFly there are a lot of index.ts files that re-export many modules. So just a simple import may be pulling a lot more than you thought. If a clean architecture is followed, this may not be a big deal, but if you combine this with the kind of reverse relationships we saw above, it amplifies the problem really quickly. Since we learnt about this, we have removed a lot of the existing index.ts files and changed most imports to be more specific.

Another reason this index.ts files are problematic is bundle splitting. Right now, the DragonFly Javascript bundle is almost 15 MBs of size. On mobile clients this is usually not a big concern as the bundle is included in the app binary and it’s downloaded only once. However, for web clients this is a real problem as it negatively impact cold start time. Similarly, it is also a problem for OTA since it forces mobile clients to download big payloads. Having index.ts files that re-export a lot of symbols make it very challenging to do tree-shaking, which is the main technique to split the bundle into smaller bundles that can be loaded at different times throught the app lifecycle.

Examples of harder to remove circular imports

Once you have separate types into a different file and remove the index.ts files, good old software engineering design principles need to be applied in order to decouple modules and subsystems so they keep their dependencies relationship to a minimum. Things like dependency injection, facade pattern, layers of indirection and many other well studied patterns are the tools to clean up the dependency graph and remove the cycles.

One example of this in DragonFly is the navigation system. On one side, React Navigation library requires to pass the page components to its stack navigation components. This means the DragonFly router has a dependency on all pages. On the other side, the pages need the navigation store to implement navigation between pages. And the navigation store also depends on the router to provide type safe mechanism to detect broken links at compile time. This creates a circular dependency that we have not figure out yet. The likely solution is to introduce a layer of indirection/abstraction.

Previous page: Hook debugging