System and method for joint source-channel encoding, with symbol, decoding and error correction

Abstract

Disclosed is a A system and method for joint source-channel encoding, symbol decoding and error correction, preferably utilizing an arithmetic encoder with operational error detection space; and a combination sequential, and arithmetic, encoded symbol decoder structure.

Description

If Δ=0.1 is chosen for the null zone magnitude, and hard decoding is performed on the received values while marking bits corresponding to signals that fall in the null zone by a “*”, then the following results:
{circumflex over (X)}_Δ=0.1={0,0,0,1,0,0,*1,*1,1,0,1,1};
and the Tree of FIG. 5 results. Decoding the received bits using an Arithmetic Decoder (4) it is found that reading the sixth bit the decodes the symbol corresponding to the error detection space. The fact that the symbol corresponding to the error detection space has been decoded is denoted in FIG. 5 with an “X”. If the current path is discarded then the decoding has to be terminated. Therefore the path is retraced to the root of the Tree and the Δ is increased so that Δ=0.2. This causes one of the signals that was originally outside the null zone to fall inside the null zone thus providing additional branch points so that the decoded bit sequence becomes:
{circumflex over (X)}_Δ=0.2={0,0, 1,1,0,0,*1,*1,1,0,1,1}

where denotes the explored branch point and the * denotes unexplored branch points.

FIG. 6 shows the new decoding Tree corresponding to said binary string. The decoding begins at the root and the error is detected as before. However, this time there exists an alternative. Instead of progressing along the upper branch, steps are retraced to the first branch point and then a path along the lower branch is pursued. The new path is denoted in FIG. 6 as a dashed line. This time , after the 10^th bit an error is detected. Hence, steps are retraced once again to the previous branch point and progress is pursued down the dotted line to find the decoded sequence.

In order to capture an error it is sufficient that Δ be greater than the magnitude of the error. It would seem then that selecting a large value for Δ is desirable, however, as already mentioned, such an approach leads to proliferation of branches in a resulting Tree. Further, it is known that small magnitude errors are more likely than are large magnitude errors, and as a result large values of Δ typically do not provide significant benefit. Also, it is noted that the probability of an error being within the last “n” symbols is:
1−(1−ε)ⁿ
and as a result it is possible to keep the default value of small, and increase it for signals corresponding to the last “n” symbols when an error is detected. Another consideration is that the probablity of an error being in a symbol close to the point at which an error is detected is higher than the probability of the error being in a symbol further away. With that in mind it is again noted that the reason for increasing the value of Δ is to increase the number of branch points and that if the number of branch points is increased too far, computational time can be wasted pursuing wrong paths. This leads to the insight that the null zone magnitude can beneficially be adjusted in a discriminating manner, and an algorithm enabling this is:

• • let Δ_inc be the amount by which Δ is increased at each step, and
  • let L_nz be the number of symbols over which it is proposed to increase the value of Δ.
    When an error is detected the path thereto is retraced for the purpose of locating a branch point. If sufficient bits corresponding to L_nz symbols are retraced then the null zone is expanded by increasing Δ by Δ_inc for all signals corresponding to the bits from the current position to the bits corresponding to the farthest symbol forward where an error was detected. Next the path is proceeded along forward with arithmetic decoding. If an error is detected within the next L_nz symbols retracing is pursued through bits corresponding to 2L_nz symbols before expanding the null zone. This time the null zone is incremented for all signals from the current position to the farthest position where an error was detected by Δ_inc. Note that this has the effect of increasing the null zone within L_nz symbols of the error detection by 2L_nz. If a correct path is still not available on the decoding tree and steps corresponding to 3L_nz have been retraced, the null zone is again incremented to all signals from the current location to the farthest point forward where an error was detected by Δ_inc. This procedure is continued until the correct path is found, or the computational burden exceeds a predetermined threshold. In the later case a decoding error is declared. By expanding the null zone in this manner the result is that null zones at different signal locations are approximately proportional to the probability of an error being detected, as shown in FIG. 7. Further, if channel characteristics are known, the probability of an error escaping the null zone can be calculated. For instance, if the channel is an AWGN, (ie. Added White Gaussian Noise), channel and the signal to noise ratio over the channel is 10 dB then the probability of a hard decision error is around 10⁻⁴. The probability of the error falling outside of a null zone with Δ=0.4 √{square root over (E_s)} is about 10⁻¹⁴. If an acceptable packet loss rate for the scheme is around 10⁻⁶ then an increase in the null zone beyond 0.4 √{square root over (E_s)} could not be justified. Because of this condition a maximum null zone, denoted as Δ_max(p), is chosen for a given probability of transmission error.

It is noted that in an arithmetic decoder an error will almost always propagate. However, the use of detection space essentially guarantees that any error will eventually be detected. The “Depth First” algorithm allows correction of the errors by exploring branches of a code tree, but said approach can become computationally expensive. It is, however, possible to prune a code tree in order to reduce the number of computations. Several constraints can be used to accomplish said pruning the code tree, and the inventors herein have made use of the fact that making incorrect “corrections” causes increased deviation from a correct path. Detection of proceeding along an incorrect path can be accomplished by, for instance, keeping track of Hamming distance, and/or keeping track of a Squared distance in the Euclidean sense.

Regarding the Hamming distance approach, keeping continuous track of the number of corrections still extant is key, with said count being compared against a threshold (T_h). The value of (T_h) is the maximum Hamming Distance between a received and decoded sequence which it is decided can be tolerated. The reasoning is that the probability of more errors is less than the probability of fewer errors, and that if an additional correction make the number of corrections extant greater than (T_h), then the null zone should be expanded by increasing the value of Δ. Expanding the null zone increases the number of possible branch points and this increases the possibilities for decoding sequences at a distance (T_h) or less from the received sequence.

Regarding the approach based on Euclidean distance, a squared distance between received and decoded symbols is monitored. A running sum of the distance between the sequential decoder output (x_k) and the received sequence is computed and compared to the distance between the output of the hard decision decoder (x_k) and the received sequence. At a time “n” this is accomplished by comparing the

Euclidean distance for the sequential decoder means:

$K_e=\frac{1}{n}\underset{k=1}{\sum ^m}{\left(r_k-{\hat{x}}_k\right)}^2$
where m is the encoded bit sequence length; with a threshold:

$T_e=\frac{1+\alpha }{N}\underset{k=1}{\sum ^N}{\left(r_k-{\hat{x}}_k\right)}^2$
where α is an experimentally determined offset. The idea is that as hard decisions are changed the Euclidean distance between decoded and received sequences increases. If the distance increases at a high rate it can be detected and is indicative of proceeding down a wrong path. If a high rate of increase is detected the decoder takes the same action as it did for the case where (T_h) is exceeded under the Hamming approach. If this approach becomes too restrictive, the offset which is considered acceptable can be incremented.

The value of (T_h) can be initialized to 1.0 if it is desired to explore all single error events, with increases in (T_h) being implemented only when a maximum value of Δ is applied. It is noted, however, that single errors with large Δ may actually be less a problem than double errors with a small Δ. Thus, it can be advantageous to increase the value of (T_h) before increasing Δ.

In view of the foregoing, it should be appreciated that there are three parameters which can be varied in controlling the discard criteria, namely:
Δ; (T_h); and α.

In the following two present invention application scenarios are discussed, namely Breadth First and Depth First. In the Depth First approach the complexity depends almost completely on the number of symbol decodings that take place during a packet decoding. For a Breadth First approach, two major factors affect the complexity. The first is the average number of decodings that take place during the decoding of a packet, which remains less than M times the number of symbols in a given packet. The second factor is the sorting that takes place before an expansion at a branch point.

With the foregoing in mind, additional comments are appropriate regarding two distinguished approaches to Decoding, (ie. Breadth First and Depth First).

Breadth First Criteria

Applying the Breadth First approach, involves fixing the size of the null zone prior to decoding. It is desirable to keep the null zone small to reduce the number of branch points, and hence the amount of computation, small. At the same time it is necessary to utilize a null zone sufficiently large that the probability of missing an error is below what it is determined can be tolerated. Assuming an AWGN channel with a known SNR, Δ can be selected as:

$\Delta \left(p,q\right)=\frac{Q^{-1}\left(1-{\left(1-q\right)}^{\frac{1}{m}}\right)}{Q^{-1}\left(p\right)}-1.$
where m is the number of bits per packet, p is the channel error probability, and q is the desired lower bound on packet decoding rate. The function Q is given by:

$Q\left(x\right)={\int }_x^{\infty }\frac{1}{\sqrt{2\pi }}{e}^{-\frac{y^2}{2}}dy.$
For this value of Δ the average number of branch points can be calculated as:

$\begin{array}{cc}B\left(p,q\right)=m{\int }_{\left(1-\eta \right)Q^{-1}\left(p\right)}^{\left(1+\eta \right)Q^{-1}\left(p\right)}\frac{1}{\sqrt{2\pi }}{e}^{-\frac{y^2}{2}}dy.& \left(2\right)\end{array}$
which simplifies to:
B(p, q)=m[Q((1−η)Q⁻¹(p))−Q((1+η)Q⁻¹(p))]  (3)
where η=Δ(p,q) and m is the average number of bits per packet. In this implementation, detection of the error detection symbol by the decoder is used to prune the code tree, and the Euclidean distance between the decoded and received sequence is used for selecting the best M paths. However, picking the value of M involves tradeoffs with larger values of M increasing the probability that a correct path will be discarded. The solution adopted was to first perform decoding using a small value of M. If this does not result in successful decoding then M is increased by a value M_inc and the procedure is repeated. Said procedure is repeated until the packet is decoded or a predetermined threshold M_max is reached, at which point a decoding error is declared.

It is further noted, in the context of a Breadth First approach, that knowing the Modulation Technique applied can allow determination of Specifc Bistable Elements in a Sequential Decoder means which serve as fixed “Branch Points”. FIG. 1b demonstrates a present invention system applied to practice a method based in the alluded to approach. FIG. 1b is much like FIG. 1a, but note that a Modulator (5) is specifically shown present between the Arithmetic Encoder (1) and Channel (2), and that the Sequential Decoder of FIG. 1a is identified as further comprising a Demodulator (Demod). Note also that a binary bit stream (x_k) enters Modulator (5), but that a modulated binary bit stream (y₁) enters the Channel (2) with the sequential signals entered to the Demod/Sequential Decoder being identified by (r₁), rather than (x_k). It is not felt necessary to provide additional Figures to aide with understanding of the effects of this. Rather, referral to FIGS. 3-5 shows that various series of sequential bits result where bits are changed at various Bistable Element “Branch Points”, and said various series of sequential bits can be demonstrated as Branches of a Tree. It is possible to, when an error is determined by detection of the presence of an unexpected, or the absence of an expected encoded reserved symbol by an Arithmetic Decoder means, to form a plurality of possible series(es) of sequential bits, followed by:

• • selection being made of a series of sequential bits, said selection being made from a group consisting of a plurality of such series of sequential bits which result from the changing of bistable elements in said sequential decoder means, (ie. the various branches of a Tree).

Of course the selected series of sequential bits will be determined by at least one criteria being met, said criteria being for instance:

• a. any series of sequential bits which contains an encoded reserved symbol is eliminated;
• b. a metric is applied to at least two series of sequential bits which do not contain an encoded reserved sysbol, to the end that determination is made regarding which of said at least two series of sequential bits is most likely correct;
• c. an Euclidean metric is applied to at least two series of sequential bits which do not contain an encoded reserved sysbol, to the end that determination is made regarding which of said at least two series of sequential bits is most likely correct. Where this approach is embodied, a suitable decoder, (ideal for use with system which utilizes Trellis Coded Modulation (TCM)), is a List Viterbi decoder where the size of the list can be adjusted to provide increased error protection, with, of course, increased complexity. The Kozintsev et al. article titled “Image Transmission Using Arithmetic Coding Based on Continuous Error Detection”, which was cited in the Background Section, it is noted, reported utilizing arithmetic coding for additional error correction in a primarily serially concatenated inner convolutional code scheme. Kozintsev et al., however, did not suggest abandoning punctured convolutional codes as the basis of the bulk of their error correction methodology.

Depth First Criteria

To implement the Depth First approach the parameters required are:

• • initial value of the null zone Δ ;
  • the value by which the null zone is to be incremented Δ_inc;
  • the “backtrack” distance L_nz;
  • the threshold to be used to prune (T_h); and
  • the parameter a to be used in constructing a threshold for Euclidean distance.

It has been found useful to define two thresholds T_h,t and T_h,w for Hamming distance and two thresholds α_t and α_w for the Euclidean distance. The total Hamming distance between the decoded sequence and the sequence obtained by hard decision decoding to the threshold T_h,t as previously described. The Hamming distance between the decoded sequence on the code Tree and the sequence obtained by hard decision decoding in a sliding window of size L_w to the threshold T_h,w. The end point of the sliding window is the current bit. A similar procedure is used for the Euclidean distance. It is noted that the values of T_{h, t} and T_h,w are obtained using two estimates of channel noise variance, one for the entire received sequence σ_t², and one for the sliding window of size L_w. The variance σ_t² is translated into a channel probability error “p”, and the two thresholds are obtained as:
T_k,f=np(1+4σ_t), T_h,w=L_wp(1+8σ_w)
In a specific case, the length of the sliding window L_w was set to 50, and both the T_h parameters were set to a minimum default value of 2. The T_e parameters were found by hard decision decoding to produce X and then setting the α_t and α_w to 0.2 and 2.0 respectively. The value of Δ was initially set to 0.10 √{square root over (E_s)}. When the decoder backtracked a symbol distance of L_nz=5, the value of Δ was increased by Δ_inc=0.10 √{square root over (E_s)} to a maximum of 0.70 √{square root over (E_s)}. If the decoder backtracked to the root of the code Tree, the values of T_h,t and T_e,t

$\text{(where:}$ $T_{e,w}=\frac{\left(1+0.2\right)}{N}\sum _N{\left(r_k-x_k\right)}^2+T_{e,t}=\frac{\left(1+0.2\right)}{L_w}\sum _{L_w}\left({\left(k-x_k\right)}^2\right)$
were increased by 10% and the values of T_h,w and T_e,w were increased by 20%.

Computational effort was determined by computing the ratio of the total number of decode operations performed by the decoder to the number of symbols transmitted. In the case where no errors occurred this ratio is one. When an error is detected, because of backtracks, the decoding scheme requires more decode operations than the number of symbols transmitted resulting in a value greater than one. When said ratio exceeded 10³ a decoding failure was declared.

FIGS. 8a and 8b present an example of the progression of the decoding procedure. FIG. 8a plots the number of symbols decoded vs. the number of decode operations. The horizontal lines represent the locations of the errors and the downward spikes indicate when the null zone is expanded. The trace progresses upward in a linear fashion until an error is detected. At that point the decoder backtracks. The is indicated by the downward slope of the decode trace. The trace continues downward until a branch point is reached. Then the trace slopes upward once again until another error is detected or the sequence is terminated. Note that there can be a lot of “jostling” around the errors until the correct branch point is found. Also note that the null zone is only expanded when the progression backtracks for a symbol distance of L_nz. FIG. 8b shows the final magnitude of the null zone denoted by the solid “stair step” line. Also shown are the error locations and magnitudes. These are the peaks along the bottom. To obtain a correct decoding, the null zone region must be expanded to encompass all the introduced errors.

Table 1 presents the results of using the depth first decoding approach in terms of packet recovery rates for the four different values of the error detection space:

• • (ε=0.5, 0.29, 0.16, 0.08).

TABLE 1
Packet Recovery Rates for Depth First Decoding
		pc = 10−1.5	pc = 10−2.0	pc = 10−2.5	pc = 10−3.0
	NONE	0.00	0.00	0.01	24.64
	ε = 0.08	0.00	0.39	46.63	96.72
	ε = 0.16	0.00	17.04	95.94	99.17
	ε = 0.29	0.00	71.09	99.21	99.56
	ε = 0.50	0.19	88.23	99.51	99.66

For comparison, also included is the case where the standard arithmetic encoder is used, albeit with packetization. The results show a more than 99% recovery rate for ε=0.16, 0.29, and 0.5; at a channel error rate of 10⁻³ Similarly high results hold for ε=0.29 and 0.5 for p_c=10^−2.5; where p_e is the probability of error for a symbol being transmitted over the channel.

However, for higher error rates the recovery rates drop significantly. Note that for a given channel error probability the amount of error space that is used is inversely proportional to the probability of packet loss.

To implement the Breadth First approach various parameter values were selected as follows: M=200, M_inc=1800, and M_max=2000. Δ was chosen to be {1.20, 1.00, 0.91, 0.82} for channel error probabilities of {10⁻¹⁵, 10⁻², 10^−2.5, 10⁻³}, respectively. The parameters used give the lower bounds on packet loss rates of {10^−1.510⁻³, 10⁻⁴, 10⁻⁵}, respectively. It should be recalled that he algorithm functions by first listing all possible paths at a branch point, then pruning all but the M which are closest in Euclidean distance, to the received sequence. Between the branch point paths get pruned because progressing along them results in the decoding of the error of the detection space.

FIG. 9 shows a plot of the fraction of paths that have not been pruned at each point. The average fraction of parallel decodings used is also indicated on FIG. 9 by the horizontal line. Note that more than half of the M paths are valid when a branch point is reached, the number of branches after the point is greater than M. The algorithm will then prune the paths furthest from the received sequence. If the valid path is ever pruned, the sequence will not decode at all or, with a small probability, it will decode incorrectly.

FIGS. 10a, 10b, 10c and 10d show how the decrease in the channel error probability, and corresponding drop in number of branch points, affects the required number of parallel decodings. Also note that the histogram for ε=0.5 shifts quite significantly, dropping to around 50%. However, the distribution for ε=0.08 is concatenated on the left and the reduction on branch points has little effect. Using M=200 to decode the packet results in being unable to decode the packet, hence the value of M was increased to 2000. If thereafter the packet could still not be decoded, a decoding failure was declared. For channel error probabilities of 10⁻³ and 10^−2.5 a higher number of paths was used for less than one packet in 20,000 for all values of the error detection space. For a channel error probability of 10⁻² the same result held for ε=0.5 and ε=0.29. For ε=0.16 the higher number of paths was used for about 20% of the packets and about 7% of the packets could not be decoded even with the higher number of paths. The situation was significantly worse for ε=0.08 with only 20% of the packets decoded using M=200. For a channel error rate of 10^−1.5 and ε=0.5, a little more that 20% of the packets were decoded using M=200 while about 50% of the packets were decoded with M=2000. For other values of almost all the packets that could be recovered required M=2000.

Table 2 presents recovery rates for the case where Breadth First decoding was applied.

TABLE 2
Packet Recovery Rates for Breadth First Decoding
		pc = 10−1.5	pc = 10−2.0	pc = 10−2.5	pc = 10−3.0
	NONE	0.00	0.00	0.01	24.64
	ε = 0.08	0.00	38.53	99.89	99.99
	ε = 0.16	0.00	92.03	99.94	99.99
	ε = 0.29	16.63	99.30	99.95	99.99
	ε = 0.50	73.24	99.33	99.95	99.99

The recovers rate is greater than 99% for all values of c for channel probabilities of error of 10^−2.5 and 10^−3.0. For p_e=10⁻² the recovery rate is still greater than 99% for ε=0.29 and 0.5. For p_c=10^−1.5 a recovery rate of 73% of the packets for ε=0.50 may still be useful for some applications. Notice that at higher error rates the Breadth first approach substantially out-performs the Depth First approach. A penalty is paid for this performance at lower error rates, however, where the computational cost of the Breadth First approach is higher than the Depth First approach.

Finally, performance of the present invention Joint Source Channel Coding Strategy is compared to that of three conventional schemes:

• • (4/5) convolutional codes, (one with 16 states and one with 8 states); and
  • (223,255) Reed Solomon Code.
    All these schemes have approximately the same rate overhead (the exact number of information bits per data bit are shown in Table 3).

TABLE 3

Coding Rates
               Rate 
  
  ⁢
  
    
      information  bits
    
    
      data  bits

NONE           1.000
ε = 0.29       0.901
ε = 0.50       0.819
(223, 255) RS  0.875
⅘ Conv, s = 8  0.800
⅘ Conv, s = 16 0.800

The performances of the identified schemes is plotted in FIG. 11. Said schemes are assumed to have the same header protection. Note that FIG. 11 shows that the packet loss rat for two convolutional cases noted are considerably worse over the simulated range. Also note that the Reed Solomon code performs worse for low signal to noise ratios (SNR), but when the SNR increases to around 5.625, the Reed Solomon marginally out-performs the present invention approach. The SNR's used in the simulations were:

{2.368, 4.323, 5.714, 6.790} decibels.

Continuing, the amount of redundancy indicated in Table 3 shows that the convolutional codes have the highest amount thereof, followed by the present invention scheme with ε=0/5. The present invention scheme with ε=0.29 has the lowest amount of added redundancy of the schemes compared. It should be specifically appreciated that the present invention algorithm is only slightly more complex than a standard Arithmetic encoding scheme, with the added complexity being present primarily at the decoder.

In the Depth First approach the complexity depends almost completely on the number of symbol decodings that take place during a packet decoding, hence the complexity is slightly more than the average number of symbol decodings for a given SNR.

As alluded to earlier, for a Breadth First approach, two major factors affect the complexity. The first is the average number of decodings that take place during the decoding of a packet, which remains less than M times the number of symbols in a given packet. The averages can be seen in FIG. 11. The second factor is the sorting that takes place before an expansion at a branch point. The average number of branch points can be calculated using Equation 3. At each branch point there exists the possibility of needing to sort if the number of current paths is more than half of the maximum. The complexity of the sort in the decoding used is Mlog₂(M). Thus, there is a large linear dependence, and an Mlog(M) dependence in the decoding complexity.

Present invention schemes provide substantial packet recovery rates at channel rates as low as 10^−1.5 with low coding overhead. Such schemes are useful in hostile communication environments where minimal coding overhead is advantageous. The approach may be especially useful for mobile and wireless applications.

The present invention can be applied in communication systems which operate based on Binary Phase Shift Keying (BPSK), Quadrature Phase Shift Keying (QPSK) and Trellis Coded Modulation (TCM) etc.

It is noted that the terminology “variable length” refers to the length of code words assigned to input symbols, and the “joint source-channel symbol encoding” refers the use of the same encoding means to encode “allowed alphabetic symbols” and “non-alphabet symbols” for use in error correction.

Finally, it is noted that the present invention is primarily useful when applied with variable length symbol coding methods. For example Huffman coding provides coding more probably symbols with shorter bit sequences. Arithmetic encoders code strings of symbols in a sequence of bits, and Claim language structure is focused to apply thereto.

Having hereby disclosed the subject matter of the present invention, it should be obvious that many modifications, substitutions, and variations of the present invention are possible in view of the teachings. It is therefore to be understood that the invention may be practiced other than as specifically described, and should be limited in its breadth and scope only by the Claims.

Claims

1. A variable length symbol, joint source-channel encoding, symbol decoding and error correction system comprising:

encoder system;

modulation-transmission means; and

combination sequential, and encoded symbol, decoding systems;

2. A variable length symbol joint source-channel encoding, symbol decoding and error correction system comprising:

encoder means;

modulation-transmission means; and

combination sequential, and encoded symbol, decoding means;

3. A joint source-channel encoding, symbol decoding and error correction system comprising:

encoder means;

modulation-transmission means; and

decoding means;

4. A joint source-channel encoding, symbol decoding and error correction system as in claim 3 in which said encoder means is an arithmetic encoder and said decoding means comprises, as the encoded symbol decoder means, an arithmetic decoder.

5. A joint source-channel encoding, symbol decoding and error correction system comprising:

encoder means;

modulation-transmission means; and

decoding means;

6. A joint source-channel encoding, symbol decoding and error correction system as in claim 5 in which said encoder means is an arithmetic encoder and said decoding means comprises, as the encoded symbol decoder means, an arithmetic decoder.

7. A joint source-channel encoding, symbol decoding and error correction system comprising:

arithmetic encoder system;

modulation-transmission means; and

decoding system;

8. A joint source-channel encoding, symbol decoding and error correction system comprising:

arithmetic encoder means;

modulation-transmission means; an

decoding means;

9. A method of correcting errors in decoded symbols which are encoded by an encoder means in a joint source-channel coding system, comprising the steps of:

a. providing a joint source-channel encoding, symbol decoding and error correction system comprising:

encoder means;

modulation-transmission means; and

decoding means;

10. A method as in claim 9, wherein said encoded reserved symbol is selected from a group of at least two reserved symbols, and wherein at least two different expected reserve symbols are selected and entered into said modulation-transmission means.

11. A method of correcting errors in decoded symbols which are encoded by an arithmetic encoder in a joint source-channel coding system, comprising the steps of:

a. providing a joint source-channel encoding, symbol decoding and error correction system comprising:

arithmetic encoder means;

modulation-transmission means; and

decoding means;

12. A method as in claim 11, wherein said encoded reserved symbol is selected from a group of at least two reserved symbols.

13. A method of correcting errors in decoded symbols which are encoded by an arithmetic encoder in a joint source-channel coding system, comprising the steps of:

a. providing a joint source-channel encoding, symbol decoding and error correction system comprising:

arithmetic encoder means;

modulation-transmission means; and

decoding means;

14. A method as in claim 13, wherein said encoded reserved symbol is selected from a group of at least two reserved symbols, and wherein at least two different expected reserve symbols are selected and entered into said modulation-transmission means.

15. A method of correcting errors in decoded symbols which are encoded by an arithmetic encoder in a joint source-channel coding system, comprises the steps of:

a. providing a joint source-channel encoding, symbol decoding and error correction, system comprising:

arithmetic encoder means;

modulation-transmission means; and

decoding means;

16. A method of correcting errors in decoded symbols as in claim 15, in which in step d. involves the determination of the presence or absence of non-alphabet symbols other than as expected, said non-alphabet symbols being not-allowed as arithmetic encoder input symbols.

17. A method of correcting errors in decoded symbols as in claim 15, which comprises practicing step d. more than once, with said error correcting method further comprising the step of:

e. defining a tolerable Hamming distance threshold Tc, and keeping count of the number Kc of “branch points” in said sequential decoder means at which correction of the “1”/(“0”) or “0”/(“1”) based binary bit thereat by inverting to “0”/(“1”) or “1”/(“0”) has been performed; and

if Kc exceeds Tc, expanding the null zone by increasing the magnitude of Δ, thereby making available additional “branch points”.

18. A method of correcting errors in decoded symbols as in claim 17, said error correction method of further comprising the step of:

f. determining in a second or greater practice of step e. if the identified “branch point” is sequentially prior to the “branch point” identified in the immediately previous practice of step e. and if so decreasing the value of Kc by 1, otherwise increasing the value of Kc by 1.

19. A method of correcting errors in decoded symbols as in claim 15, which comprises practicing step d. more than once, with said error correcting method further comprising the step of:

e. defining a means for calculating a euclidean distance between received and decoded symbols, and a tolerable rate of increase of euclidean distance between sequential practice of step d., and

if said euclidean distance increases faster than at said tolerable rate, expanding the null zone by increasing he magnitude of Δ, thereby making available additional “branch points”.

20. A method of correcting errors in decoded symbols which are encoded by an arithmetic encoder in a joint source-channel coding system, comprises the step of:

a. providing a joint source-channel encoding, symbol decoding and error correction system comprising:

arithmetic encoder means;

modulation-transmission means; and

deocding means;

21. A system, comprising:

a decoder, comprising an arithmetic decoder and a sequential decoder, configured to receive data from an arithmetic encoder via a channel,

wherein the arithmetic decoder, upon detecting an absence of an expected encoded reserved symbol or a presence of an unexpected encoded reserved symbol, is configured to initiate an error correction routine,

wherein the arithmetic decoder is configured to not initiate the error correction routine upon detection of an allowed symbol in the received data, and

wherein the sequential decoder is configured to, upon initiation of the error correction routine by the arithmetic decoder, change at least one bistable element in the sequential decoder or select a series of sequential bits from a plurality of series of sequential bits which result from the changing of bistable elements in the sequential decoder.

22. The system of claim 21, further comprising:

the arithmetic encoder configured to receive input data; and

a modulator located between the arithmetic encoder and the channel.

23. The system of claim 21, wherein the decoder is configured for use in a communication system.

24. The system of claim 23, wherein the communication system is configured to use Binary Phase Shift Keying (BPSK), Quadrature Phase Shift Keying (QPSK), or Trellis Coded Modulation (TCM).

25. A method, comprising:

receiving data from an encoder via a channel at a decoder, the decoder comprising an arithmetic decoder and a sequential decoder; and

producing output data using the decoder,

wherein the arithmetic decoder, upon detecting an absence of an expected encoded reserved symbol or a presence of an unexpected encoded reserved symbol, initiates an error correction routine,

wherein the arithmetic decoder does not initiate the error correction routine upon detection of an allowed symbol in the received data, and

wherein the sequential decoder, upon initiation of the error correction routine by the arithmetic decoder, changes at least one bistable element in the sequential decoder or selects a series of sequential bits from a plurality of series of sequential bits which result from the changing of bistable elements in the sequential decoder.

26. The method of claim 25, further comprising:

receiving coded data from the encoder at a modulator, which outputs modulated data to the channel.

27. The method of claim 25, wherein the received data is a signal corresponding to a binary bit stream sequence of “1”/(“0”)'s and “0”/(“1”)'s.

28. The method of claim 27, wherein the producing output data further comprises making hard logic circuitry decisions as to a presence of the “1”/(“0”)'s and “0”/(“1”)'s based on the signal.

29. The method of claim 28, wherein the producing output data further comprises using the sequential decoder to monitor output from the arithmetic decoder for errors.

30. The method of claim 29, wherein the producing output data further comprises:

identifying a branch point if the signal has a value located within a null zone; and

correcting the “1”/(“0”) or “0”/(“1”) binary bit by inverting it to “0”/(“1”) or “1”/(“0”) respectively, at the sequential decoder if an output from the arithmetic decoder indicates an error.

31. The method of claim 30, wherein the identifying a branch point and the correcting the “1”/(“0”) or “0”/(“1”) binary bit is performed more than once.

32. The method of claim 31, wherein the producing output data further comprises determining a number of branch points at which correction of the “1”/(“0”) or “0”/(“1”) binary bit has been performed (Kc).

33. The method of claim 32, wherein the producing output data further comprises:

decreasing a value of Kc by 1 if a subsequent branch point is sequentially prior to a first branch point; and

increasing the value of Kc by 1 if the subsequent branch point is not sequentially prior to the first branch point.

34. The method of claim 32, further comprising:

determining a Hamming distance threshold (Tc); and

expanding the null zone to increase a number of branch points, if the number of branch points at which correction has been performed (Kc) exceeds the Hamming distance threshold (Tc).

35. The method of claim 31, wherein the producing the output data further comprises:

determining a euclidean distance between the received data and the output data; and

expanding the null zone to increase a number of branch points, if the euclidean distance for subsequent calculations increases faster than a defined rate.

36. The method of claim 25, wherein the producing the output data further comprises:

selecting a series of sequential bits in the sequential decoder by,

eliminating any series of sequential bits which contains an encoded reserved symbol;

applying a metric to at least two series of sequential bits which do not contain an encoded reserved symbol and determining which series is most likely correct; or

applying an euclidean metric to at least two series of sequential bits which do not contain an encoded reserved symbol and determining which series is most likely correct.

37. The method of claim 25, further comprising using the decoder in a communication system.

38. The method of claim 37, further comprising using Binary Phase Shift Keying (BPSK), Quadrature Phase Shift Keying (QPSK), or Trellis Coded Modulation (TCM) in the communication system.

39. A system, comprising:

a decoder, comprising an encoded symbol decoder and a sequential decoder, configured to receive data from an arithmetic encoder via a channel,

wherein the encoded symbol decoder, upon detecting an absence of an expected encoded reserved symbol or a presence of an unexpected encoded reserved symbol, is configured to initiate an error correction routine,

wherein the encoded symbol decoder is configured to not initiate the error correction routine upon detection of an allowed symbol in the received data, and

wherein the sequential decoder is configured to, upon initiation of the error correction routine by the encoded symbol decoder, change at least one bistable element in the sequential decoder or select a series of sequential bits from a plurality of series of sequential bits which result from the changing of bistable elements in the sequential decoder.

40. The system of claim 39, wherein the encoded symbol decoder is an arithmetic decoder.

41. The system of claim 39, further comprising:

the arithmetic encoder configured to receive input data; and

a modulator located between the encoder and the channel.

42. The system of claim 39, wherein the decoder is configured for use in a communication system.

43. The system of claim 39, wherein the communication system is configured to use Binary Phase Shift Keying (BPSK), Quadrature Phase Shift Keying (QPSK), or Trellis Coded Modulation (TCM).

44. A method, comprising:

receiving data from an encoder via a channel at a decoder, comprising an encoded symbol decoder and a sequential decoder; and

producing output data using the decoder,

wherein the encoded symbol decoder, upon detecting an absence of an expected encoded reserved symbol or a presence of an unexpected encoded reserved symbol, initiates an error correction routine,

wherein the encoded symbol decoder does not initiate the error correction routine upon detection of an allowed symbol in the received data, and

wherein the sequential decoder, upon initiation of the error correction routine by the encoded symbol decoder, changes at least one bistable element in the sequential decoder or selects a series of sequential bits from a plurality of series of sequential bits which result from the changing of bistable elements in the sequential decoder.

45. The method of claim 44, further comprising:

receiving the data from the encoder via a modulator, which outputs data to the channel.

46. The method of claim 44, further comprising using an arithmetic encoder as the encoder and an arithmetic decoder as the encoded symbol decoder.

47. The method of claim 44, further comprising using a signal corresponding to a binary bit stream sequence of “1”/(“0”)'s and “0”/(“1”)'s as the received data.

48. The method of claim 47, wherein the producing output data further comprises making hard logic circuitry decisions as to a presence of the “1”/(“0”)'s and “0”/(“1”)'s based on the signal.

49. The method of claim 48, wherein the producing output data further comprises using the sequential decoder to monitor output from the encoded symbol decoder for errors.

50. The method of claim 49, wherein the producing output data further comprises:

identifying a branch point if the signal has a value located within a null zone; and

correcting the “1”/(“0”) or “0”/(“1”) binary bit by inverting it to “0”/(“1”) or “1”/(“0”) respectively, in the sequential decoder if an output from the encoded symbol decoder indicates an error.

51. The method of claim 50, wherein the identifying a branch point and the correcting the “1”/(“0”) or “0”/(“1”) binary bit is performed more than once.

52. The method of claim 51, wherein the producing output data further comprises determining a number of branch points at which correction of the “1”/(“0”) or “0”/(“1”) binary bit has been performed (Kc).

53. The method of claim 52, wherein the producing output data further comprises:

decreasing a value of Kc by 1 if a subsequent branch point is sequentially prior to a first branch point; and

increasing the value of Kc by 1 if the subsequent branch point is not sequentially prior to the first branch point.

54. The method of claim 52, further comprising:

determining a Hamming distance threshold (Tc); and

expanding the null zone to increase a number of branch points, if the number of branch points at which correction has been performed (Kc) exceeds the Hamming distance threshold (Tc).

55. The method of claim 51, wherein the producing output data further comprises:

determining a euclidean distance between the received data and the output data; and

expanding the null zone to increase a number of branch points, if the euclidean distance for subsequent calculations increases faster than a defined rate.

56. The method of claim 51, wherein the producing the output data further comprises:

selecting a series of sequential bits in the sequential decoder by,

eliminating any series of sequential bits which contains an encoded reserved symbol;

applying a metric to at least two series of sequential bits which do not contain an encoded reserved symbol and determining which series is most likely correct; or

applying an euclidean metric to at least two series of sequential bits which do not contain an encoded reserved symbol and determining which series is most likely correct.

57. The method of claim 44, further comprising using the decoder in a communication system.

58. The method of claim 57, further comprising using Binary Phase Shift Keying (BPSK), Quadrature Phase Shift Keying (QPSK), or Trellis Coded Modulation (TCM) in the communication system.