The term "heterodox"--like, say, "infidel"--is necessarily imprecise; it categorizes people by what they don't believe rather than what they do. In the case of heterodox economists, what they don't believe is the neoclassical model that anchors the economics profession.
I was in a department that was considered Heterodox,. In fact, my adviser is quoted in this article. It is strange to be defined by what you don't believe. I suppose heterodox are sort of like atheists - in their belief in the Truth of Not believing they suddenly sound exactly like the people that don't believe in.
The common definition of Neoclassical economics taken from Lionel Robbins is the science which describes human behaviour as a relationship between (given) ends and scarce means which have alternative uses. What Neoclassical Economics proposes is that we are all Utility Maximizers and Rational-decision makers with access to Perfect information. Behind these definitions are, what is called, Consumption and Production Functions that make up Supply and Demand Curves that determine something else called General Equilibrium. Neoclassical Economics proposes that markets tend toward Equilibrium, except in the case of market failure. There are many agreed upon instances of Market Failure, such as Monopoly power, Externalities, Public goods and Common resources, but for most markets, this model accurately describes how markets work and price is determined.
Graduate school in economics involves coursework that revolves around the mathematical equations that produce these terms in real values. Thus Price is Value and is a Real term represented by a number we can compare to other numbers or value. All of this reveals economics to be different from all the other social sciences because it has Mathematical validity and equations that support its theory and is thus considered by most economists and many others to be a Science. There is an elegance behind these equations that reveal Truth or Value in such real terms. It is very easy to be enamored by them. Heterodox economists are no less enamored by them than Orthodox economists.
There are many assumptions underlying the Neoclassical model and Utility Maximizer and Rational Decision makers with Perfect information are among the ones that Heterodox economist focus upon. What if humans are not Utility Maximizers (We're not)? What if we are not Rational Decision makers (We aren't)? And what if we do not have access to Perfect information (We don't)? Heterodox economists focus on these questions and spend their time doing empirical studies revealing that these assumptions and others are false.
They then tweak the model and attempt to write equations that more closely resemble how humans make decisions and come up with alternative equations for consumption and production functions so they can still determine General Equilibrium and their work will still be considered Science by their peers in economics and fit under the ever increasing rubric called Neoclassical Economics. The problem with their work is that the equations get increasingly complex and the heterodox economist loses the elegance that was the main attraction to Orthodox economics in the first place.
Milton Friedman, the orthodox economists figurehead, long ago admitted that many of the assumptions underlying neoclassical economics were false. In his essay The Methodology of Positive Economics, he argued that it was not important that humans were rational, but rather that they behaved as if they were rational in the aggregate and that the model should not be judged on the accuracy of its assumptions but rather on how well its performs. The problem with heterodox economists is that none of their models perform as well as the orthodox model.
So what is the problem with economics? Economics by itself is not the problem. Neoclassical economists proposed a model that is dominant within the field and now, to do economics, one must be adept at mathematics. Alfred Marshall, who first coined the term neoclassical economics, defined economics as a study of mankind in the ordinary business of life. This definition is much less specific and accurately describes how economists approached their discipline around the fin de siecle. Economists were located in philosophy departments and their papers were filled with rhetoric instead of equations as they tried to determine how humans attained happiness in life and the role that markets and money played in this pursuit.
The problem with orthodox and heterodox economists is their treatment of economics as a science, instead of a discpline that is searching for the answers to some fundamental questions such as "What are humans for?" and whether or not our quest for profits at the expense of our physical and mental healths is really in our best interests. Economics should have rhetorical arguments on the nature of our modern existence and whether or not we are really achieving any success in terms of freedom and happiness, as we go about our ordinary business in life. Neither Heterodox or orthodox economists are asking the right questions as they focus on how we make decisions by describing agents in abstract models taking their influence and formulation directly from the physical sciences, instead of what decisions we should be making that best preserve and benefit humanity and its future here or anywhere on the planet.