Title
Search
All Issues
419 
Year: 2007  Vol. 11   Num. 2  - Abr/Jun - (11º) Print:
Section: Original Article
Versão em PDF PDF Português Texto Text in Portuguese
Evaluating the Benefits of Digital Hearing Aids and Perceptions of Hearing Deficiencies in Non-institutionalized Elderly
Author(s):
Maria Helena Pinho Costa1, André Luiz Lopes Sampaio2, Carlos Augusto C. P. de Oliveira3
Key words:
Presbyacusia. Aging. Hearing aids.
Abstract:

Introduction: Individuals of more than 60 years of age may experience difficulty inspeaking over the telephone or where noise levels are high, or may require higher volume to understand television and radio programs. The hearing aid has become a primary therapeutic option for persons with hearing deficiencies that cannot be resolved through clinical or surgical treatment. With recent technological advances, most people with hearing deficiencies can benefit from using a hearing aid. The benefits provided by the use of amplification have been defined as the difference between a person's performance with and without a digital sound amplification device (DSAD). Objectives: Subjectively evaluate the benefits derived from the digital sound amplification device and assess perceptions of the psychosocial effects of hearing impairments. Methods: Thirty elderly users of DSAD in the 60-89 year age bracket suffering from bilateral postlingual neurosensory hearing deficiencies participated in the study. Evaluation of benefits was based on the questionnaire Abbreviated Profile of Hearing Aid Benefit - APHAB, under conditions with and without the hearing device. Evaluation of perception of the hearing disadvantage was done through the questionnaire Hearing Handicap for the Elderly - HHIE. Results: Average of 63.8% of the answers indicated benefits from the use of DSAD and 80% showed some degree of perception of the hearing impairment. Conclusion: Benefits were registered in subjective evaluations, with significant improvement in communications through use of the DSAD and perceptions of hearing impairments in the evaluated group.

INTRODUCTION

Loss of hearing sensitiveness occurs, as years go by, due to genetic predisposition or to exogenous factors, such as exposure to intense noise, use of ototoxic medication and presence of some co-morbidities. Frequently, individuals in the 50-60-year age bracket present difficulty to communicate on the phone or in noisier places or, still, need to listen to louder television or radio in order to understand messages. Such symptoms are usually related to presbyacusis, neurosensory hearing loss associated to aging mainly affecting high frequencies and harming word discrimination and thus communication. The most common complaints refer to difficulty to understand what patients have been told (1).

According to World Health Organization (WHO), hearing impairment means reduction of sensitiveness or hearing discrimination, difficulty locating sounds or tinnitus (2). Such impairment may be measured through psychoacoustic techniques, such as audiometry, speech perception test and evaluation of auditory processing or through electrophysiological techniques, such as the investigation of Evoked Response Audiometry of central auditory ways or the investigation of evoked otoacoustic emissions. Hearing impairment is observed as such deficiency affects performance and ability of individuals to perform daily routine activities, such as speech or environment sound perception. The handicap is, therefore, defined as the impact which such impairment causes in the well-being and life quality of individuals, and reverberates in inter-personal relations, in emotional balance, interactions or social, educational and occupational aspirations (2).

In the elderly, the Digital Sound Amplification Device (DSAD) is the prime mover element of hearing rehabilitation. The selection of the best hearing aid, as well as use guidance, is challenges to professionals, such as speech therapists and otorhinolaryngologists. Currently, DSAD are individualized equipment, which is miniaturized, digital or programmable, versatile and has controls which can be easily manipulated by users, even the oldest ones.

DSAD1s benefits may de defined as the advantages or gains and even profits obtained by users originated from sound amplification (3). Such benefit may be measured, and considered positive, negative or even neutral, depending on the effect that DSAD has on the performance of each individual (4). The benefit may be exclusively related to relief of the sensation of peripheral hearing loss, or may occur regardless of users' perception, favored by the improvement of their audition in daily activities (3, 5). Consequently, the benefit brought to users of sound amplification varies among individuals, mainly due to their motivation to use sound amplification (6). Therefore, a subjective clinical evaluation of patients regarding their perception of hearing impairment is important, that is, the impact which such auditory privation brings to patients every day, that is, the handicap.

In audiology, the benefit offered by DSAD users may be measured by means of the difference between users' performance in situations with and without DSAD, in evaluations done in identical conditions (6, 7). Such benefit may also be measured through objective evaluations - functional gain and speech recognition tests - and subjective evaluations, with self-evaluation questionnaires, comparing situation with and without DSAD (8, 9).

The isolated objective evaluation of the benefit originated from DSAD use is not enough, once the functional gain is verified in acoustic cabins, with pure tones and phonetically balanced words, which cannot be compared, in any ways, to daily situations. Consequently, on one hand, it is important that the benefit be evaluated in daily situations, when environmental sound also compete with human voice; on the other hand, the DSAD noise reducer may identify the pure tone as an environmental noise in grave frequencies and the evaluated gain becomes insufficient. In such situation, patients may present some benefits with sound amplification which may be detected through subjective methods, even if the gain which has been objectively evaluated is not enough.

Therefore, the questionnaire Profile of Hearing Aid Benefit (PHAB) was created by COX e RIVERA in 1992 (10) in order to evaluate the benefit provided by DSAD use in two different situations with and without DSAD. However, , in 1995, in clinics, COX and ALEXANDER (11) developed a reduced version of such questionnaire, the Abbreviated Profile of Hearing Aid Benefit (APHAB), which has been translated to Portuguese by Almeida (12) as Benefício do Aparelho de Amplificação Sonora, in 1998.

APHAB (Annex 1) is a self-evaluation questionnaire in which individuals quantifies difficulties experienced in different routine communication situations. It is considered an effective instrument to provide standardized data in order to evaluate the hearing incapacity associated to hearing impairment. It is made of 24 questioned grouped in subscales which refer to speech recognition in daily environments in which the following aspects are evaluated: communication easiness (CE) in environments without competitive sounds, reverberation (RV), which refers to listening quality in big spaces, and environmental noise (EN), which evaluated the capacity to recognize speech within competitive noise, in addition to reaction of users to environmental sounds (ES), which verifies aversion of users to sounds, not the benefit itself. The three first scales evaluated speech recognition in three different situations of daily life and the last one quantifies negative reactions to environmental sounds. In such subscale, the discomfort level to sounds with DSAD use is evaluated, not the benefit itself.




Such questionnaire, when answered in different situations - with and without DSAD use - allows measuring DSAD benefit. All items are statements concerning communication abilities or sound perceptions in daily situations and the individuals under test should indicate how frequently each statement is true. A seven-point scale is provided and a descriptive term and a percentage are related to each answer option, which are "always"(99%), "almost always"(87%), "usually" (75%), "half of times" (50%), "sometimes"(25%), "rarely"(12%) and "never"(1%). The higher the value of each subscale is, the higher the difficulty experienced in daily communication situations. Each item must be answered twice, one for condition without DSAD and the other for condition with DSAD. APHAB evaluated both isolated individual performance, with or without DSAD, and the benefit provided by the amplifier, and the difference between both situations is recorded. Such questionnaire is considered a clinical instrument to validate results obtained with sound amplification which may be useful to quantify individuals' impairment related to hearing loss and its reduction with amplification use.

Handicap or hearing disadvantage may also be evaluated through questionnaire application. The questionnaire Hearing Handicap Inventory for the Elderly (HHIE) was developed by VENTRY and WEINSTEIN, in 1982 (13) and translated to Portuguese by WIESELBERG, in 1997 (14) (Annex 2). This questionnaire was developed to evaluate the auditory and non-auditory - psychosocial - effects caused by hearing impairment in the elderly. This instrument may be used in the process of hearing rehabilitation, to verify the perceptive reduction of hearing disadvantage, to indicate the use of sound amplification and, still, as a complement to perform audiometric triage (5). In order to be used as fast evaluation procedure in elderly patients with hearing disorders, HHIE has a simplified version called HHIE-S. Such questionnaire version was validated in 1988 by BESS, LICHTENSTEIN and LOGAN (15).




HHIE is made of 25 questions, out of which 13 explore the emotional consequences of auditory impairment and 12 refer to social and situational effects of such deficiency. Therefore, the handicap is evaluated according to scoring criteria. In 1982, VENTRY and WEINSTEIN (13) proposed a way to measure hearing disadvantage: four points to each "yes", two points to each "sometimes" and zero points to "no". Consequently, zero point means the absence of hearing disadvantage, which is higher as high as the individual's perception concerning their hearing disadvantage, that is, difficulties originated from auditory impairment.

This study aims at subjectively evaluating the benefit of digital hearing aids in non-institutionalized elderly patients and identifying their perception concerning psychosocial and situational effects originated from hearing disadvantage caused by hearing impairment.


CASUISTICS AND METHOD

A prospective study was performed in a convenience sample of 30 non-institutionalized elderly patients, who were 60 years old or older and were under ambulatory otorhinolaryngological and audiological treatment in a private clinic located in Brasilia - DF, between July and October 2004. 30 consecutive patients who were older than 60 years, literate, with neurosensory hearing loss acquired after language acquisition, under use of bilateral digital DSAD for at least 3 months and for at most 24 months, with average of 11.2 months.

All patients underwent previous otorhinolaryngological evaluation and had indication of DSAD use. Such patients were cared by the same professionals, one speech therapist and one otorhinolaryngologist, during the process of indicating, selecting and adapting the DSAD. The evaluation protocol used in this study was submitted and approved in the research ethics committee of Medicine School of Universidade de Brasilia, under number 061/2003. The members of this research sign a free and clear consent term, according to ethical principles to perform investigations in human beings (Annex 3).




All patients answered, in paper, two different self-evaluation questionnaires, without help of other people. After reading the texts, each individual chose the best option. To verify the benefit of DSAD use, the self-evaluation questionnaire APHAB was used, in ALMEIDA's version (12) in two different situations - with and without DSAD. The general benefit was calculated through the differences among the scores averages with and without DSAD of each subscale questionnaire. The subscale of sound aversion was used to verify the discomfort level to sounds with the use of DSAD and not for benefit calculation, calculated and separately presented for each answer given to each item of each subscale which makes the questionnaire. Data were stratified according to benefit percentage in lower than - 50%, from - 22% to - 10%, from - 9% to + 9%, from + 10% to + 22%, from + 23% to + 49% and higher than + 50%.

The benefit may be negative in some questions, once the users may do a better evaluation without DSAD use concerning the situation with its use. Consequently, the highest negative percentage corresponds to the highest difficulty degree, while the highest positive percentage refers to the highest benefit. To consider that there was benefit with DSAD use, we used the 22% cut point among the results obtained with and without DSAD, and from 23% on we considered that there was a benefit from subjective point of view (11).

Hearing disadvantage was verified through a self-evaluation questionnaire HHIE, in version by WIESELBERG (1987) (Annex 2) (14), which observed the effects of hearing loss in emotional and social aspects, in non-institutionalized elderly patients. Such questionnaire was applied just for the situation with DSAD use. Patients were than stratified according to their perception of hearing loss with DSAD use in: <= 16 (without perception), 17 to 42 (slight perception) and > 42 (severe perception) (14).

Individuals were classified according to the degree of hearing loss, considering the best ear, according to classification by DAVIS and SILVERMAN, developed in 1970 (16), which uses the mathematical average of responses in audiometric frequencies 500, 1000 e 2000 Hz, and which classifies the degrees of auditory loss for mathematical averages in: mild, from 26 to 40 dB NA; moderate, from 41 to 70 dB NA; severe, from 71 to 90 dB NA and deep, higher than 90 dB NA.


RESULTS

All 30 patients answered both questionnaires and there was no individual exclusion in this study. Individuals selected for this study were using three different models of digital DSAD of same manufacturer. Out of the 30 (100%) caregivers, 24 (80%) were female and 6 (20%) were male. Subjects aged between 60 and 89 years, out of whom 7 (23%) were in the 60-69 year age bracket; 9 (30%), in the 70-79 year age bracket and 14 (47%) in the 80-89 year age bracket. When they were stratified concerning hearing loss, considering the best ear, 13 (43.3%) presented mild hearing loss and 17 (56.7%) presented moderate neurosensory hearing loss. Just one patient presented severe hearing loss in the worst ear and mild one in the best ear.

Table 1 presents the distribution of 180 answers given by patients to APHAB questionnaire for the six items related to CE. In 1% (n=2) of answers, the benefit rate varied from -9% to +9%. The benefit was negative and varied from -50% to -10% in 4.3% of answers (n=8), and, in 20% of answers, varied from 10% and 22%. There was subjective perception of the benefit with DSAD use - benefit rate > 22.0% - therefore, in 74.7 percent of answers.




Table 2 shows distribution of responses related to RV. Out of the responses, 48.7% (n=87) show reverberation interference in communication quality with DSAD uses and benefit, therefore insufficient - benefit brackets lower than 23% - and 11% (n=20) presented benefits lower than -5-%, 2.8% (n=5) varied from -22 to -10%, 13.9% (n=25) from -9 +9%. There was benefit for such subscale in 51.7% of responses evaluated (n=93).




Concerning EN subscale (Table 3), it has been observed that, in 35% of responses given (n=63), there was no benefit registered among patients, and 7.8% of responses (n=14) presented benefits lower than -50%; 5.0% (n=09) varied from -22 to -10%, 7.2% (n=13) from -9 +9% and, in 65% of responses (n=117), there was benefits higher than 22%.




Concerning ES, Table 4, it is realized that, with aid use, 46.2% (n=83) of reports of disconfort to sounds were reported.




The general benefit was generally verified in 63.8% of responses evaluated, taking into account that for CE subscale it was 74.7%, EN 65% and RV 51.7% of evaluated responses.

When the perception of hearing impairment was evaluated by patients - handicap - social losses were noticed by 48% of patients, while emotional losses were noticed by ///52% of patients. Hearing impairment was noticed in 48% of situations evaluated, adding up "yes" and "sometimes" responses.

When the perception of hearing impairment was quantified, it was observed that just 20% of responses indicated non-perception of hearing impairment by patients. In 80% of other responses, patients stated perception of hearing impairment in mild to severe degree, according to Table 5.




DISCUSSION

According to RUSSO (17), hearing loss is possibly the most devastating disorder in aging process. Presbyacusis is usually the factor which signs third age bringing difficulties in communication and generating important sequels of emotional, social and occupational nature. DSAD is a recommended resource which enables speech recognition. However, the use of such devices not always provides users with normal hearing, or with the one similar to the one before presbyacusis, mainly in situations in which there is competitive noise.

According to SILVEIRA and RUSSO (18), conventional audiological evaluation can provide data related to kind and degree of auditory deficiency which patients present, and the evaluation of how such deficiency affects emotional and social quality of life of patients through other instruments is also essential. With such objective, self-evaluation questionnaires have been more used in order to evaluate social and emotional consequences of hearing impairment. Such questionnaires are, therefore, important instruments to clinical routine to verify the benefits of the use of hearing aid concerning speech recognition in silent environments, noisy environments or reverberating environments in addition to the disadvantage originated from hearing impairment. In addition to this, they are instruments which complement objective evaluation, such as functional gain, insertion gains and speech recognition tests.

It is known that adaptation difficulties of hearing aids in the elderly are related to the degree of hearing loss, to speech recognition, to difficulties in handling DSAD volume control, to motivation to use DSAD and to accepting hearing loss, as well as to expectations related to benefits of hearing aids. It is frequent to observe older patients who also have little social insertion presenting higher resistance to hearing aids, once they consider it is not economically and socially worth investing money in such an expensive device. In such cases, only conventional objective evaluations are not enough to evaluate and conduct the DSAD adaptation process.

As the questionnaires are translations from other languages, it is necessary to have the adaptation of such instruments to the social, cultural, mental and cognitive level of the patient group evaluated. In this investigation, it has been observed that, during the questionnaires, some patients wanted to know what was being evaluated. It is important to point out the need to develop evaluation instrument which are originally proper to our social, cultural and economic reality.

Moderate hearing losses represent 56.7% of sample studies and mild hearing losses represent 43.3%. Patients with moderate hearing loss present higher communication difficulty when message is emitted in low intensity, according to the distance and competitive noise. In clinical practice, a non-proportional harm between speech recognition and the degree of hearing loss is frequently observed in elderly patients. Consequently, losses that are initially considered mild may be translated into deep harms of central hearing system, that is, intrinsic complementary redundancies which are inherent to central nervous system tend to reduce as age comes, according to RUSSO (19). Such fact may explain benefits which are not to satisfactory of DSADS use in populations with mild hearing loss verified in the present study.

Around 74% of responses revealed benefit in the CE subscale with DSAD use, so it was the subscale which revealed benefit with the highest frequency. The result verified for silent environments suggests that hearing impairment in such patients probably occurs not only due to a hearing reduction, which can be completely corrected through amplification, but also due to possible alterations in central hearing process. Therefore, the treatment of such patients, followed by orientations and training which are more aimed at such difficulty, must be motivated by professionals who deal with such kind of treatment.

However, RV subscale presented benefit in lower frequency of responses, 51.7%. Such result is certainly due to the acoustics of the environment and to the distance between speaker and listener, considering that it is easier to recognize speech in less reverberating environments than in reverberating environments, which makes communication more difficult. A similar result was found by FERRARI (20), BUCUVIC (21) and by ASSAYAG (22).

Concerning the difficulty to recognize speech in noisy environments, EN subscale, a benefit with hearing aid was observed in 65%. Elderly patients usually find speech recognition difficulty, mainly in noisy environments. Such finding also agrees with presbyacusis characteristics - lowering in high frequencies - which make communication in noisy environments.

In ES subscale, which emphasizes users' impression before intensity of environmental sounds, high rates of discomfort reports with DSAD were found, 46.2% of given answers. Such result is expected in DSAD users, once increase of hearing of acoustic signals occurs making them more intense with amplification and disturbing, in varied ways, DSAD users.

Average benefit obtained among three subscales (CE, RV and EN) was verified in 63.8% of responses given.

In the present study, patients with adaptation period between 3 months and 24 months with average of 11.2 months were evaluated. Consequently, benefit could be observed in higher number of responses if the adaptation period were higher, mainly in patients who have been using it for three months (n=2). Such results, according to scientific literature, show benefits with DSAD use. In 1988, ALMEIDA (10), verified favorable statistically significant difference with DSAD use, in 2003 (22), verified global benefit of 51.7% with DSAD.

The impairment caused by hearing impairment may be expressed in social and emotional situations. In this study, 48% of patients related disadvantage in social scope and 52% revealed emotional impairment. Perception of disadvantage was considered severe according to the methodology used in 40% of answers. Perception of hearing impairment directly influences the benefit of DSAD use, one, the higher the perception of disadvantage is the more the individuals will wear DSAD and realize gains or advantages with DSAD.

It is important to point out that the questionnaire which evaluates hearing impairment was applied with DSAD use. Consequently, perception of hearing impairment before DSAD use could be higher, once some patients (n=16) have been using DSAD for more than one year.

According to WIESELBERG, in 1997, (14) hearing impairment diminishes as patients get used to DSAD, thus reducing the disadvantage caused by hearing loss and reintegrating patients in their social life. Therefore, we point out the need of a systematic treatment of patients who wear DSAD, in order to be reevaluated concerning adaptation and benefit, identifying difficulties and offering phonoaudiological guidance related each case, in order to reduce the index of people who quit wearing DSAD.

The use of APHAB questionnaire may evidence some difficulties presented by patients with DSAD use in several listening situations, enabling support through discussion of such difficulties and adopting communication strategies which make patients use the signals they listen to in a better way.


CONCLUSION

Before the results of present investigation, we can reach the following conclusions: there was an average benefit in around 63.8% of the responses given by patients in such subscales evaluated and there is perception of hearing impairment in 80% of patients in the evaluated group.

The subjective evaluation of hearing aids is important to validate the results of amplification and should be incorporated to clinical routine of speech therapists. Data point out the importance of subjective approach in individual evaluation of psychosocial consequences caused by hearing impairment, enabling strategies for its reduction.


REFERENCES

1. Russo ICP. Uso de próteses auditivas em idosos portadores de presbiacusia: indicação, adaptação e efetividade. São Paulo; 1988, p. 33. Tese de Doutorado. Escola Paulista de Medicina - Universidade Federal de São Paulo.

2. OMS. International classification of impairments, disabilities and handicaps: a manual of classification relating to the consequences of disease. Genebra: World Health Organization; 1980.

3. Jerger J. What determines benefit from hearing aid? J am acad audiol; 2000, v. 11(8), p.368.

4. Humes LE. Dimensions of hearing aid outcome. J am acad audiol; 1999, v. 10, p. 26-39.

5. Weinstein BE, Ventry IM. Audiometric correlates of hearing handicap inventory for the elderly. J Speech Hear Disord; 1983, v. 48, p.379-384.

6. Almeida K, Iório MCM. Próteses auditivas: Fundamentos teóricos e aplicações clínicas. 2ª ed. São Paulo: Lovise; 2003.

7. Weinstein BE. Geriatric Hearing loss: myths, realities, resources for physicians. Geriatrics; 1989, v. 44(4), p.42-59.

8. Russo ICP, Almeida K, Freire KGM. Seleção e adaptação da prótese auditiva para o idoso. In: Almeida K, Iório MCM.

Próteses auditivas: fundamentos teóricos e aplicações clínicas. 2ª ed. São Paulo: Lovise; 2003, p.385-410.

9. Matas CG, Iório MCM. A avaliação do desempenho das próteses auditivas. In: Almeida K; Iório MCM. Próteses auditivas: fundamentos teóricos e aplicações clínicas. São Paulo: Lovise; 2003, p.305-334.

10. Cox RM, Rivera IM. Predictability and reliability of hearing aid benefit measured using the PHAB. J. Am. Acad. Audiol.; 1992, v. 3, p.242-254.

11. Cox RM, Alexander GC. The abbreviated profile of hearing aid benefit. Ear Hear; 1995, v. 16, p.176-186.

12. Almeida K. Avaliação objetiva e subjetiva do benefício das próteses auditivas em adultos. São Paulo; 1998, p.20. Tese de Doutorado. Escola Paulista de Medicina - Universidade Federal de São Paulo.

13. Ventry IM, Weinstein BE. The hearing handicap inventory for the elderly: a new tool. Ear Hear; 1982, v. 3(3), p.128-134.

14. Wieselberg MB. A auto-avaliação do handicap em idosos portadores de deficiência auditiva: o uso do HHIE. São Paulo; 1997, p. 30. Dissertação de Mestrado. Departamento de Fonoaudiologia - Pontifícia Universidade Católica de São Paulo.

15. Bess FH, Lichtenstein MJ, Logan SA. Making hearing impairment functionally relevant: linkages with hearing disability and handicap. Acta Otolaryng. suppl.; 1991, v. 476, p.226-231.

16. Davis H, Silverman SR. Hearing and deafness. 4th ed. New York, Holt, Rinehart and Winston, 1970.

17. Russo ICP. Perfil global do idoso candidato ao uso de prótese auditiva. Rev. Pró-Fono; 1999, Set/Dez, p.426.

18.Silveira KMM, Russo ICP. A percepção da deficiência auditiva por idosos institucionalizados. Rev. Soc. Bras. Fonoaudiol.; 1998, p.05-11.

19. Russo ICP. Distúrbios da audição: a presbiacusia. In: Russo I.C.P. Intervenção fonoaudiológica na terceira idade. Rio de Janeiro: Revinter; 1999, p.51-82.

20. Ferrari DV. Aparelhos de amplificação sonora individual: caracterização e utilização em adultos com deficiência auditiva neurossensorial. São Paulo; 1999, p. 22. Dissertação de Mestrado. Departamento de Fonoaudiologia - Pontifícia Universidade Católica de São Paulo.

21. Bucuvic EC. Avaliação subjetiva das dificuldades auditivas e do benefício da amplificação em pacientes novos usuários de aparelho auditivo. São Paulo; 2003, p. 34-35. Dissertação de Mestrado. Escola Paulista de Medicina - Universidade Federal de São Paulo.

22. Assayag FHM. Avaliação subjetiva do benefício e efeitos proporcionados pelo uso de amplificação sonora em indivíduos idosos. São Paulo; 2003, p. 28. Dissertação de Mestrado. Departamento de Fonoaudiologia - Pontifícia Universidade Católica de São Paulo.










1. Master degree in Medical Science by Universidade de Brasíla (Owner & Manager of the Audiológica Aparelhos Auditivos)
2. PhD in Health Science by Universidade de Brasíla (Physician of the Otorhinolaryngology department of the Universidade de Brasíla)
3. PhD (Professor of the Otorhinolaryngology discipline of the Universidade de Brasíla)

Universidade de Brasília
SEP/S 715/915 CJ A BL D SL 412/413 Centro Clínico Pacini Asa Sul Brasília/DF CEP. 70390-155

This article was submitted to SGP (Sistema de Gestão de Publicações) of R@IO on March 20th, 2007 (code #231) and approved on June 11th, 2007.
  Print:

 

All right reserved. Prohibited the reproduction of papers
without previous authorization of FORL © 1997- 2024