Entropy, How it was discovered
An essential concept for the modern world
The significance of the steam engine is not appreciated. Mostly regarded as an obsolete contraption, this invention was already powering commerce before it was understood how it actually worked! Way back when in Britain (1700s), they used horses, donkeys, or mules to run crude pumps to get water out of underground mines. Someone figured out that with coal mines, it might be smart to burn coal to drive the pumps instead. Ventilation was also an issue. These coal mines were terrible places, and about to get worse, and are expertly described as they were in the 1930s by George Orwell in his book The Road to Wigan Pier.
People knew about the incredible power of steam long before, but hadn’t figured out how to effectively use it. A piston in a cylinder is the basic element, back and forth motion converted to rotary motion with a crankshaft was the ultimate solution, although there were other methods. Machining a steam engine accurately enough so it wouldn’t wear itself out prematurely, and figuring out a way to control the application of steam through a governor were the two big advances that made it work, usually credited to James Watt (a Scot). His last name has become the standard unit for power, the time rate of energy.
But even at that point, no one understood how it really worked. One thing that was recognized is that heat traveled always from hot to cold, never the other way ‘round. At that time, the exact nature of heat was also poorly understood. But people plowed ahead.
Another effort involved trying to use the waste heat from combustion to get more power from the engine. In fact, many were trying to make this thing drive itself with the waste heat, a perpetual motion machine. So, how to tie energy, heat, and temperature together to figure this out?
One thing that had already been explained mathematically was that energy could exist in various forms, such as fluid velocity and pressure. This idea suggests that energy is conserved in such transformations; there’s a certain amount of energy in the universe and all it does is transform into various forms, such as velocity (kinetic energy), potential energy (water at the top of a waterfall), or pressure. That is really the first step in understanding, and came to be called the First Law of Thermodynamics—energy is conserved, and not created or destroyed. Without further thought, one might conclude the universe is an eternal clock, running forever. That certainly seems to be the attitude here on Earth, with all of our schemes to use energy, at no cost, since it can neither be created or destroyed.
In those days, instrumentation wasn’t that easy to do, so a lot of things were worked out with thought experiments, and this is one of the best examples ever. A few observations were made:
1. Heat only travels from hot to cold, never the other way
2. A device that only transfers heat from hot to cold, but does no work on the environment is impossible.
3. A device that only does work on the environment, but transfers no heat, is impossible.
4. Items 2 and 3 are equivalent, and are expressions of the Second Law of Thermodynamics.
5. All real world devices have friction and other sources of losses.
Examples of such devices are refrigerators, heat engines (such as a steam engine), pumps, and compressors. Item 5 brought about the idea of a thought experiment using a perfect device, not actually possible to build, that has no friction or other sources of losses, and any heat transferred was done across an impossibly small temperature difference. The idea is that a frictionless device like that, if operated slowly enough, could be reversible, so that the original state (pressures, temperatures, amount of mass) could be regained by simply reversing the process. If that modeling was useful, it would at least set a bound on what was possible with real world devices.
A heat engine takes heat (another form of energy) from somewhere (very familiar is taking heat from combustion, for example), and performs work.[1] Its thermal efficiency is the energy of the work divided by the energy of the heat input. If you have two reversible heat engines with the same heat input temperature and the same heat sink temperature, they have the same thermal efficiency. The thermal efficiency of a real world heat engine can never exceed the thermal efficiency of the reversible one. And conveniently enough, using the absolute temperature scale, the reversible thermal efficiency is entirely defined by the temperature of the source and the temperature of the sink (by their ratio, actually).[2] This sets an upper bound to what any conceivable real world heat engine can achieve for thermal efficiency.
All sorts of tricks have been employed to improve thermal efficiencies in the name of Save the Planet and other well-intentioned efforts, such as increasing the temperature ratio by raising the source temperature more and more, or decreasing the sink temperature. But if you’re using atmospheric air, which contains almost 80% nitrogen (N2), the higher temperature produces more oxides of nitrogen (NOx), a pollutant. This is an example of the sort of thing that happens when you try to short-circuit nature, namely the Second Law of Thermodynamics. Not only is there a cost for energy transformations, there is a cost for increased efficiencies and the cost goes up at an increasingly accelerated rate the closer to the theoretical limit you get.
The numbered list above contains a form of the Second Law, in items 2 and 3. It can also be stated that all the heat input into a heat engine cannot be converted into useful work. There’s always “waste” heat, in other words.
The classic proof of this concept entails two heat engines, one reversible, the other not, with the non-reversible claimed as efficient or more than the reversible one. They provide the same amount of work. The source for both is the same temperature, as well as the same temperature for the sink of both. If the non-reversible one was more efficient, it would generate more work than the reversible one, which is impossible, given its irreversibilities of friction, etc. that would create more heat and absorb energy from that available to do work. If the non-reversible one was as efficient as the reversible one, it would have to be reversible, also, another contradiction to the claim. The two contradictions are taken as disproving the original claim, leaving the only other conclusion, that the reversible one is always most efficient, given the same environmental conditions.
It seems like a torturous way to prove something, but it makes sense considering a reversible process is used to define another state property, entropy. Entropy is yet another mathematical construct, like energy, and descriptively is a way to show that even if energy cannot be created or destroyed, there is still a cost to any energy transformation. This is the reason it’s worth having some understanding of this whole topic, as a lot of marketing, social theory and “scientific” promotion typically glosses over this detail.
The way this was originally proven is another thought experiment, using a reversible, cyclic device. The “cycle” is a very general term, but think of some device that could flow one way in its cycle, and flow the other way in the rest of its cycle and ends up at its beginning state when the cycle is complete, as an example. It’s clear they were thinking of steam engines, as a cyclic device, but it turns out that is not the particular type of device used to demonstrate this, as will be seen. This device receives heat from a constant temperature source, TH, let’s say.[3] It performs work. And finally, it rejects heat during its cycle to some other system that is not necessarily at a constant temperature, so is a variable, T.[4] An arbitrary boundary can be set around the device and the other system receiving the “waste” heat. Note there is no heat passed from the other system receiving the heat from the device that goes across the boundary, although there could be work from the other system going outside the boundary. So the source heat QH and the total work[5] W both cross the arbitrary boundary.[6]
As this is a cyclic device, instances during the cycle can be identified, where the sum of all the instances is the total net of QH and W for the entire cycle. All the instances are increments, or changes, of an arbitrarily small quantity.
Since this is reversible, the expression in endnote 2 applies, and can be rearranged to state the change in QH/TH = the change in Q/T, the change in Q and the value of temperature T, at any instant in the cycle. Conservation of energy then dictates that the net change in work incrementally through the cycle is the difference between the change in QH and any other heat loss or energy change (zero in this case), so the change in W at any instant is the same as the change in QH. Substituting the change in W (going around the cycle) for the change in QH into the rearranged expression results in the change in W = TH[(the change in Q)/T]. Being increments, these can be totaled from the beginning of the cycle to its end. (Don’t forget this is an ideal, reversible cycle) If you can show all the changes around the cycle in W are negative, or at most zero, then all of the changes in Q/T would also be negative or zero, since TH is a positive constant. That’s the next step.
In the previous paragraph, this was described as a combination of a cyclic device and a system at temperature T surrounded by a boundary, but this whole contraption within the boundary can be temporarily visualized as a composite device (black box?), coupled with a thermal energy reservoir, originally described at TH.
This is still a cyclic device, so the total net work around the cycle has to be summed all around the cycle, from where it starts back to where it starts again. There are four possibilities here: the heat transfer can be to or from the device and work can be done on or by the device. The only combination that doesn’t violate the First or Second Law is where there is work into the device (negative W), and positive Q into the thermal reservoir, summed around the cycle.[7] Reverting to the original model surrounded by the boundary, the sum of the changes in one cycle Q/T is less than or equal to zero, since TH is a constant and the sum of the changes of W is less than or equal to zero.
That’s a pretty far-reaching conclusion, and lead to basing the definition of the change in entropy at any instant as the (change in Q)/T, at any instant during the cycle of a reversible device.[8] With the previous inequality, you can see how this thought experiment set a bound on this new property entropy, based on a reversible cyclic device. Also, as this is a state property, even though it was proven using a cyclic path, it is independent of path. This is why a reversible path can be used to describe state properties on an irreversible path, to put it another way. This was the big discovery of what was called classical thermodynamics, which over time matured into a universally applicable approach to any energy transformation. The details outlined in this paragraph are linked here: Path to point, Entropy
It was rapidly concluded from this that a perpetual motion machine is impossible to make, and with this new state property of gases, a complete mathematical analysis of the conditions, say inside a steam engine, was possible. As a wonderful example, Rudolf Diesel used the ideal thermodynamic cycle originally proposed by Sadi Carnot to form a basis for his revolutionary engine, the first time this theoretical concept was used in a practical application with engines. He had worked with refrigeration before that, which gave him the mathematical tools, inclination, and experience to apply it that way.
At the time, there was no agreement on the kinetic theory of gases yet, which was later coupled with statistical methods to form a much more broad definition of entropy that includes this first, more specific definition. In any case, think of entropy as the cost of energy transformation, even though energy itself can’t be created or destroyed. Another interpretation is entropy is a tendency towards disorder, irrevocably.
Continued: Path to point, Entropy
[1] The physical definition of work is force X distance, and is the basis for defining energy. So, if you push a weight across the floor, it will take a certain amount of force over whatever distance you pushed it. The work is the product of the two. On the other hand, if you push a certain force on a wall that doesn’t move, there is no work.
[2] Let Q represent heat from the source or into the sink, and T the absolute temperature of the source or sink. For a reversible system, then QL/QH = -TL/TH. As efficiency is defined as 1 + QL/QH, it also is 1- TL/TH, for a reversible heat engine.
[3] “Receive” is used loosely here, as it’s not known yet what is the direction of heat flow. Assuming “receive” is positive, but calculation reveals a negative number, it means the heat flows the opposite way of the original assumption. So instead of receiving heat, it would emit heat. The same applies to the direction of work.
[4] This temperature is usually taken as the temperature of the environment around the device, and can vary, depending on time and location relative to the device. In abstract terms, it is just referred to as a “system.”
[5] This work could be the net total of the work the device produces that goes across the boundary, and the work the heat-receiving other system produces that crosses the boundary.
[6] A valuable exercise is for the reader to try sketching this description out on a piece of paper. Use arrows to show direction of work and heat.
[7] Think of a bad reversible refrigerator that provides no cooling! Two of these combinations have energy in the form of heat or work going into the device (not possible, 1st Law) or energy in the form of heat or work coming out of the device (not possible, 1st Law). The third combination is heat from the reservoir, through the device, and out as work, but that violates Item 3 in the list of observations, so only one option is left as possible, work applied to the device, resulting in heat out of the device and into the reservoir represented at TH. Therefore the sum of the changes of W is less than or equal to zero. It’s the only heat engine configuration of the four proposed that has a single W and single Q, that is two legs, since the other heat engine configurations have a Qsource, a Qsink, and W, three legs, that would make it not possible to conclude anything.
[8] This Q is an increment, a change—all the increments add up to the total around the cycle, which is zero, as it has to end at the same state it started at (it’s reversible). There are special ways to write that, for those interested: This increment Q/T at any point in the cycle is the change in entropy at the same place in the cycle. This change is more commonly written as dS, the differential of S, entropy. The differential of a cyclic variable can be written as δ, so dS ≡ δQ/T , for a reversible cycle. (The triple bars mean defined as.) A very useful rearrangement of this definition leads to Tds = δQ and is the first step in which this state property is related to other state properties, such as temperature, pressure, or specific volume, because δQ also equals δw + du by the First Law, where u represents internal energy. It’s also clear the invention of calculus, with its differentials invented by Leibniz and Newton, greatly facilitates this result. Finally, this definition was derived from this specific thought experiment, but is generally applicable as the classical definition of entropy.