Most projects have some previous code written. As programmers, we have to figure out where and when to refactor and how much is necessary to bring something up to the new standards. We inherited a D3 project from a few years back and went to work on refactoring some of the code. The project is called TSOMI{:}, which stands for “The Sphere of My Influence”.
The purpose of TSOMI is to use directional data from DBpedia{:} to create a playful interface so users can visually explore how their favorite artists, writers, musicians, scientists, thinkers, and theorists influence one another. We wanted to utilize Wikipedia’s powerful crowdsourcing tool to better add to and preserve this data.
TSOMI included a challenge unique to our team in that it blends React and D3. React takes control of the DOM and encourages the user to think of all components as total throwaways. D3 takes control of the DOM and wants the user to use its language to manipulate a cluster of persistent components. In other words, both libraries want total control over the DOM, creating two conflicting paradigms that must both be managed in an application. The trick behind making them work together is to strictly separate the DOM into regions that each system controls.
We reached the point at which we could not fulfill requests without significantly refactoring the rendering system. The visual component that renders our subjects interacted directly with the module that retrieves data from DBpedia and with the module that caches data about those subjects. The interaction proved too difficult to trace.
Over several weeks, we separated the rendering component from the retrieval and storage components. Because of the degree to which these components depended on one another, this ultimately became a complete rewrite of all three components.
Paradigm Mismatch
Most of this project follows user interface patterns common to most web applications, such as buttons, text fields, search widgets, and an about page. As such, we used React for the governing framework and restricted D3 to the drawing area.
Typical React rendering code involves creating and returning a set of React elements:
This gives the general feel of a pure functional transformation from component properties and state to visual elements on the screen.
Typical D3 code is exactly the opposite. In the following block, this.nodesElem refers to a D3 Selection, which itself is a wrapper around a real DOM element.
D3 operations consist of functions that make changes directly to elements in the DOM.
This is the core of the behavioral mismatch. We wanted to bridge the mismatch between React and D3’s relationship to the DOM. We created two objects, the InfluenceCanvas and the InfluenceChart. InfluenceCanvas provides the semantic concept of a display for a group of people who have influenced one another, and does so in the form of a traditional object in which methods mutate internal state over the lifetime of the object. InfluenceChart is a React component which plugs into React’s normal lifecycle methods to capture and respond to events from the browser and a data store.
Handling the D3 Component
InfluenceCanvas wraps around the D3 library, isolating direct DOM manipulation within a more semantic interface for the InfluenceChart to consume. It transforms a table containing information about people with their influences, a particular “focus person”, and the dimensions of a drawing area into an animated SVG diagram of that group of people, the links between them, and their lifespans on a timeline. It provides semantic functions, such as setDimensions and setFocused, so the outside application can simply tell the canvas when to change state. In contrast to a React component, this follows a traditional object-oriented approach to programming, in that we create and then statefully manipulate the object through its methods.
InfluenceCanvas is not completely self-contained, as it exists to manipulate DOM components. Since it interacts directly with the DOM, isolation becomes a key point. Isolation requires that we be able to search for components both by ID and by type without risking retrieving components from other areas of the DOM. The easiest way to accomplish this is to give to the InfluenceCanvas’s constructor a selection for the top of the drawing area. This lets us limit the scope for all DOM searches to only the elements within the drawing area.
We take this approach one step further. SVG images render their components from first to last, which draws the last elements in front of all of the other elements. Visually, we have a natural stack with thumbnails at the front and lifelines all the way in the back. This is convenient for grouping the three types of visual elements. In the InfluenceCanvas constructor, we save a reference to each visual group as we create it. These references allows us to scope element searches to the exact element type we need.
The React Component
InfluenceChart is the React component that wraps around InfluenceCanvas, providing all of the larger-scale application context. This is where we can handle the impedance mismatch between the event-based world of React and the direct manipulation world of D3.
In this component, render does much less than normal: nothing more than creating the SVG element that serves as the home for the D3 drawing surface.
The label specified here has to be determined by whichever component handles application layout.
The domElem and d3Elem refer to real DOM elements that the D3 underlying system needs, and importantly, they do not exist when InfluenceChart gets created or render gets called for the first time.
We get the components later. componentsDidMount is called after the DOM elements have been created. In this function, our top priority is to capture the DOM representation of the SVG that render returns, both as an HTMLElement, which we need for detecting size changes in the drawing area, and as a D3 Selection.
componentDidMount needs to create the InfluenceCanvas if the focus person has been selected and fully loaded. If we skip that here, the canvas will not be created until some other event triggers a call into getDerivedStateFromProps, leaving the canvas uncreated and the page blank. In the event that the person has not been loaded, a later event, in this application from the Redux store, will trigger a call to getDerivedStateFromProps which will handle creating the InfluenceCanvas.
Since all of the DOM elements now exist, it is important to register a callback to detect window resize events.
React does not provide a way to ask for the DOM element that corresponds to a component, and D3 does not provide a way to create a Selection from a DOM element. We can only solve this by using features of the DOM API and the D3 API to search for the elements by ID. It is important to ensure that the element has a unique ID so that both the DOM and D3 searches will return only one unambiguous result.
Where componentDidMount handles the beginning of the life of the DOM elements, getDerivedStateFromProps handles converting React and Redux events into InfluenceCanvas actions. The InfluenceChart pays attention to property updates, which like in any other React/Redux application, are hooked into the Redux store. A change in properties will cause a re-render, and so render is one place in which we could calculate the updated state. At the same time, that is not a particularly natural place to “change state”. Additionally, we have found cases in which property changes do not reliably trigger a render, but very reliably trigger a call to getDerivedStateFromProps.
Normally, getDerivedStateFromProps would be used to update this.state. In this particular use case, almost all of the interesting state occurs in the canvas, and so it is very easy to simply use the property changes to dispatch to the InfluenceCanvas.
We need one final element, componentDidUpdate. Changes to DOM properties, such as we use to restyle the drawing area after the Wikipedia component has been hidden or shown, do not trigger a call to getDerivedStateFromProps. We need to update the InfluenceCanvas’s dimensions whenever the Wikipedia component is shown or hidden. At first, we thought that we could manage this in render, but we kept getting the old size of the component, not the new size. This is similar to the problem of needing to get the initial size, because render runs long before the new layout is reliably complete.
componentDidUpdate runs after layout is complete, but not after every layout change. Importantly, though, it runs after a DOM property change has been reflected on the page, and so at this point, we can get the new element sizes and communicate them to the D3 component.
Conclusion
Cloud City has found working on this project to be challenging in that there were many frustrating “gotchas” and misadventures as we worked through the learning curve. We all entered the project with no prior experience with D3, but we emerged with a working understanding of it, plus a pattern that shows how to give a React interface to any module which uses intensive DOM access.
We are delighted to be able to now present this pattern as a tool in constructing this visualization that we find fun, fascinating, and informative.